docker model launch llmfit #747

Activity

1. ericcurtin

   added

   good first issueGood for newcomersGood for newcomers

   on Mar 14, 2026
2. ericcurtin commented on Mar 14, 2026

   

   ericcurtin

   on Mar 14, 2026

   Last edited by ericcurtin

   ContributorAuthor

   More actions

   Tagging @AlexsJones and @three-foxes-in-a-trenchcoat for awareness. Probably want to point at Docker Model Runner rather than Ollama also.
3. ericcurtin

   changed the title [-]docker model fit[/-] [+]docker model launch llmfit[/+] on Mar 15, 2026
4. sathiraumesh commented on Apr 5, 2026

   

   sathiraumesh

   on Apr 5, 2026

   Contributor

   More actions

   hi @ericcurtin can I look into this? I saw that llmfit has already included dmr as a provider. Maybe I can try to integrate it

   👍React with 👍1Reacted by Alex Jones
5. sathiraumesh commented on Apr 5, 2026

   

   sathiraumesh

   on Apr 5, 2026

   Last edited by sathiraumesh

   Contributor

   More actions

   Hi @ericcurtin @AlexsJones i was testing this localy. When
   I set DOCKER_MODEL_RUNNER_HOST=http://model-runner.docker.internal and watched Docker model logs for incoming requests which means it uses the DMR backend.

   
   docker run --rm -it \
     -e DOCKER_MODEL_RUNNER_HOST=http://model-runner.docker.internal \
     --entrypoint sh \
     ghcr.io/alexsjones/llmfit -c "llmfit"
   

   In the default cli mode, it doesn't call the dmr backend, probably pointing to Ollama

   docker run --rm \
     -e DOCKER_MODEL_RUNNER_HOST=http://model-runner.docker.internal \
     ghcr.io/alexsjones/llmfit recommend -n 3 
   

   Should we wait for this to be available in default CLI mode in llmfit?
6. AlexsJones

   added a commit that references this issue on Apr 5, 2026

   fix: query local providers in `recommend` CLI command

   ...

   This issue will close once commit 2477d1d is merged into the 'main' branch.

   2477d1d
7. AlexsJones commented on Apr 5, 2026

   

   AlexsJones

   on Apr 5, 2026

   More actions

   Thanks for testing this @sathiraumesh! You're right — the DOCKER_MODEL_RUNNER_HOST env var was only being honored by llmfit's TUI mode; the CLI recommend subcommand wasn't probing any local provider at all.

   Just cut a release that fixes this: v0.9.1. The recommend command now queries Ollama, MLX, llama.cpp, Docker Model Runner, and LM Studio — same as the TUI — so DMR will receive a request (and the installed status shows up in both text and JSON output).

   Try the same command again with the new image and you should see the request land in Docker model logs.

   👍React with 👍1Reacted by Sathira Umesh
8. sathiraumesh commented on Apr 5, 2026

   

   sathiraumesh

   on Apr 5, 2026

   Contributor

   More actions

   Thanks @AlexsJones Ill test it
9. sathiraumesh commented on Apr 6, 2026

   

   sathiraumesh

   on Apr 6, 2026

   Last edited by sathiraumesh

   Contributor

   More actions

   #837 @AlexsJones @ericcurtin you can review. I was thinking, should we allow interactive, maybe as another pr refrencing another issue
10. AlexsJones commented on Apr 6, 2026

    

    AlexsJones

    on Apr 6, 2026

    More actions

    Would docker run --rm -it ghcr.io/alexsjones/llmfit "" work to get the TUI going for now?
    I am reluctant to create another image just for a custom CMD
11. sathiraumesh commented on Apr 7, 2026

    

    sathiraumesh

    on Apr 7, 2026

    Last edited by sathiraumesh

    Contributor

    More actions

    @AlexsJones i will try to test. I think we probably need to change the existing implementation a bit to run interactively from DMR side. But I will test and see if there are any other changes that we might need
12. sathiraumesh commented on May 20, 2026

    

    sathiraumesh

    on May 20, 2026

    Contributor

    More actions

    hi @AlexsJones i created a issue request here for the llmfit project to finalize the implementation here based on my findings and comments for this feature here. Can you have a look on this when you have time