undefined | Better HN

0 pointsthrdbndndn2y ago0 comments

For the client API it's pretty clear:

    from ollama import Client
    client = Client(host='http://localhost:11434')

But I don't quite get how the example in "Usage" can work:

    import ollama
    response = ollama.chat(model='llama2', messages=[
    {
        'role': 'user',
        'content': 'Why is the sky blue?',
    },
    ])
    print(response['message']['content'])

Since there is no parameter for host and/or port.

0 comments

jmorgan2y ago

Once you have a custom `client` you can use it in place of `ollama`. For example:

  client = Client(host='http://my.ollama.host:11434')
  response = client.chat(model='llama2', messages=[...])

thrdbndndnOP2y ago

Thanks. I don't have the service installed on my computer RN, but I assume the former works because it by default uses a host (localhost) and port number that is also the default for ollma service?

flakes2y ago

Exactly that. Client host options default, https://github.com/ollama/ollama-python/blob/main/ollama/_cl...

Also overrideable with OLLAMA_HOST env var. The default imported functions are then based off of a no-arg constructed client https://github.com/ollama/ollama-python/blob/main/ollama/__i...

    # ollama-python/ollama/__init__.py
    _client = Client()

    generate = _client.generate
    chat = _client.chat
    embeddings = _client.embeddings
    ...

j / k navigate · click thread line to collapse