mirror of
https://github.com/danny-avila/LibreChat.git
synced 2026-03-08 09:02:36 +01:00
🍎 feat: Apple MLX as Known Endpoint (#2580)
* add integration with Apple MLX * fix: apple icon + image mkd link --------- Co-authored-by: “Extremys” <“Extremys@email.com”> Co-authored-by: Danny Avila <danny@librechat.ai>
This commit is contained in:
parent
0e50c07e3f
commit
d21a05606e
8 changed files with 77 additions and 0 deletions
|
|
@ -271,6 +271,40 @@ Some of the endpoints are marked as **Known,** which means they might have speci
|
|||
|
||||

|
||||
|
||||
## Apple MLX
|
||||
> MLX API key: ignored - [MLX OpenAI Compatibility](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/SERVER.md)
|
||||
|
||||
**Notes:**
|
||||
|
||||
- **Known:** icon provided.
|
||||
|
||||
- API is mostly strict with unrecognized parameters.
|
||||
- Support only one model at a time, otherwise you'll need to run a different endpoint with a different `baseURL`.
|
||||
|
||||
```yaml
|
||||
- name: "MLX"
|
||||
apiKey: "mlx"
|
||||
baseURL: "http://localhost:8080/v1/"
|
||||
models:
|
||||
default: [
|
||||
"Meta-Llama-3-8B-Instruct-4bit"
|
||||
]
|
||||
fetch: false # fetching list of models is not supported
|
||||
titleConvo: true
|
||||
titleModel: "current_model"
|
||||
summarize: false
|
||||
summaryModel: "current_model"
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "Apple MLX"
|
||||
addParams:
|
||||
max_tokens: 2000
|
||||
"stop": [
|
||||
"<|eot_id|>"
|
||||
]
|
||||
```
|
||||
|
||||

|
||||
|
||||
## Ollama
|
||||
> Ollama API key: Required but ignored - [Ollama OpenAI Compatibility](https://github.com/ollama/ollama/blob/main/docs/openai.md)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue