I ran the Deepseek r1:8b model (smallest) on my local Macbook machine.
Tuning it for basic customer support of cross-border payments with Cryptocurrency.
And did the same on ChatGPT UI and OpenAI API to test answers, for a side-by-side comparison.
Highlights
Answer Quality
I liked the reasoning aspect of Deepseek R1, which explains the logical details behind the answering.
It helps in fine-tuning the AI agents and debugging for better responses.
This option is easily collapsible, so we can show/hide it to end users.
I used the distilled model which is resource-efficient and has limited reasoning abilities.
But the full model can give us:
▶ Multi-step problem-solving
▶ Better reasoning
▶ Self-verification capabilities
Speed
Running this on my local machine (Apple M1 Pro 16GB), even using the smallest model - R1.b slowed everything on my system.
Development on local 16GB is not optimal for Gen AI training and inference, so I understand.
So this is easily solvable when we move to cloud hosting on AWS or GCP instances. As well use Deepseek's bigger models.
Cost
It is free. We don’t have to worry about tokens during training and inference.
When deployed for application AI models, this can help us drastically bring down the cost of production and subscription fees.
Security
I have not dug much into this part yet, but soon will.
Given B2B and Fintech use cases require tight customer data security and privacy, this is critical.
Currently, Orin uses hashtag#OpenAPI for Fintech AI agents apps, but anything with customer deployment using hashtag#Deepseek will need to be scrutinized for many months, passing SOC2 and other compliance standards.
Setup
Please refer to the blog post in link
