... | @@ -2,7 +2,7 @@ To facilitate the use of Large Language Models we now have an API service which |
... | @@ -2,7 +2,7 @@ To facilitate the use of Large Language Models we now have an API service which |
|
|
|
|
|
**Usage**
|
|
**Usage**
|
|
|
|
|
|
* POST requests to the API are to be JSON-encoded strings requiring a text key and allowing for the flexible addition of any other keyword arguments that can be added to the vLLM sampling params found [here](https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py)
|
|
* POST requests to the API are to be JSON-encoded strings requiring a text field and allowing for the flexible addition of any other keyword arguments that can be added to the vLLM sampling params found [here](https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py)
|
|
* Additionally output can be streamed with the boolean kwarg 'stream' and logits can be returned using the boolean kwarg 'include_logits'
|
|
* Additionally output can be streamed with the boolean kwarg 'stream' and logits can be returned using the boolean kwarg 'include_logits'
|
|
* For completion tasks you can prepend the prompt used with the boolean kwarg 'include_prompt'
|
|
* For completion tasks you can prepend the prompt used with the boolean kwarg 'include_prompt'
|
|
* A convenience function as well as default sampling configurations can be found in [this repository](https://github.com/Parry-Parry/idaLLM/tree/main/idallm)
|
|
* A convenience function as well as default sampling configurations can be found in [this repository](https://github.com/Parry-Parry/idaLLM/tree/main/idallm)
|
... | @@ -16,4 +16,3 @@ Any models currently being served will be noted here, along with their correspon |
... | @@ -16,4 +16,3 @@ Any models currently being served will be noted here, along with their correspon |
|
| Model | URL |
|
|
| Model | URL |
|
|
| ------ | ------ |
|
|
| ------ | ------ |
|
|
| Llama 2 (7B) | http://llama2api-ir.ida.dcs.gla.ac.uk/ | |
|
| Llama 2 (7B) | http://llama2api-ir.ida.dcs.gla.ac.uk/ | |
|
|
|
\ No newline at end of file |
|
| | | |
|
|
|
\ No newline at end of file |
|
|