Question: GPU/Docker Execution and Tool/MCP Integration Support #342

Activity

1. codelion commented on Dec 10, 2025

   

   codelion

   on Dec 10, 2025

   Member

   More actions

   GPU/Docker Execution

   LLM Operations

   OpenEvolve supports any OpenAI-compatible API endpoint via the api_base configuration. You can point to:

   • Self-hosted GPU servers running vLLM, TGI, Ollama, or similar
   • Docker containers with OpenAI-compatible APIs
   • Cloud GPU endpoints (Together AI, Fireworks, etc.)

   Example config:

   llm:
     api_base: "http://your-gpu-server:8000/v1"
     api_key: "your-key"
     models:
       - name: "your-model"
         weight: 1.0

   Evaluation on GPU/Docker

   The evaluator is fully customizable. Your evaluator.py can call remote GPU resources. For your MoE architecture discovery use case:

   def evaluate(program_path):
       # Read the candidate architecture
       with open(program_path) as f:
           architecture_code = f.read()
   
       # Submit to your GPU cluster/Docker for training
       job_id = submit_training_job(architecture_code, gpu_cluster_url)
       results = wait_for_job(job_id)
   
       return {
           "accuracy": results["accuracy"],
           "efficiency": results["params"] / results["flops"],
           "combined_score": compute_final_score(results)
       }
   

   Configure longer timeouts for training jobs:

   evaluator:
     timeout: 3600  # 1 hour for training jobs
   

   Tool/MCP Integration

   You can use https://github.com/algorithmicsuperintelligence/optillm as a proxy to enable tool calling and MCP server integration with any model. OptiLLM sits between OpenEvolve and your LLM, adding capabilities like:

   • Tool/function calling support
   • MCP server connections
   • Various inference optimization techniques

   Setup:

   1. Configure MCP servers in ~/.optillm/mcp_config.json:

   {
     "servers": {
       "my-gpu-tools": {
         "transport": "sse",
         "url": "http://your-mcp-server:8080/sse",
         "headers": {
           "Authorization": "Bearer ${API_TOKEN}"
         }
       }
     }
   }
   

   2. Point OpenEvolve to OptiLLM and use the mcp- prefix:

   llm:
     api_base: "http://localhost:8000/v1"  # OptiLLM proxy
     models:
       - name: "mcp-gpt-4o"  # mcp prefix enables MCP plugin
         weight: 1.0
   

   This allows the LLM to call external tools and MCP servers during code generation.

   👍React with 👍1Reacted by Xu Lingrui
2. codelion

   added

   questionFurther information is requestedFurther information is requested

   on Dec 10, 2025
3. passing2961 commented on Dec 15, 2025

   

   passing2961

   on Dec 15, 2025

   Last edited by passing2961

   Author

   More actions

   @codelion Thank you very much for your kind and detailed response—it was extremely helpful.

   I have one further question. What if the task or question involves training GPT-2? In that case, we would need to prepare a Docker container with GPU support. While it is possible to rely on cloud GPU services for fine-tuning—since many providers now offer managed fine-tuning (e.g., Together AI)—what if we instead want to train the model on my local server, which has two A100 GPUs (40GB each)?

   Is this setup supported? If so, could you explain how to set it up once again?
4. codelion commented on Dec 15, 2025

   

   codelion

   on Dec 15, 2025

   Member

   More actions

   I think there are couple of options:

   1. We can run OpenEvolve directly on your GPU server

   The simplest approach - install OpenEvolve on the machine with the A100s. Your evaluator can directly use the local GPUs:

   def evaluate(program_path):
       import torch
       # Your training code runs directly on local GPUs
       device = "cuda" if torch.cuda.is_available() else "cpu"
       # ... train GPT-2, return metrics
   

   2. Remote submission from a different machine

   If OpenEvolve runs elsewhere, set up a simple job API on your GPU server:

   # evaluator.py (runs on OpenEvolve machine)
   def evaluate(program_path):
       import requests
       with open(program_path) as f:
           code = f.read()
   
       # Submit to your GPU server's API
       response = requests.post(
           "http://your-a100-server:5000/train",
           json={"code": code}
       )
       return response.json()  # {"accuracy": ..., "loss": ...}
   

   On your A100 server, run a Flask/FastAPI endpoint that receives code, trains, and returns results.

   👍React with 👍1Reacted by Young-Jun Lee
5. passing2961 commented on Dec 15, 2025

   

   passing2961

   on Dec 15, 2025

   Author

   More actions

   Both solutions are really helpful to my current ongoing research, where I try to expand new OpenEvolve tasks that require to use GPU servers. Really appreciate it. Especially, I love the second way.

   ❤️React with ❤️1Reacted by Asankhaya Sharma
6. passing2961

   closed this as completedon Dec 15, 2025