Skip to main content

Groq - AI Api with free basic credits for testing

Feature RequestAPI

GROQ is ai infrastructure company providing api which can be used as an additional provider in EMR.

They have free tier as well which is good for testing project/demo with clients before launching with openai

Groq's APIs are openai compatible and so easier to integrate with base url:

https://api.groq.com/openai/v1

Posted by Salman Farish ·
3 comments

Log in to comment and vote

Comments3

  • Salman Farish

    •

    Jul 22, 2024

    Current request Limits of GROQ API is enough for majority of us :

  • Leon

    •

    Jul 20, 2024

    Would be great to give clients a way to respond to reviews with AI for free. As it allows for unlimited usage with a 20 API request limit per minute. So each of our customers could use their own API key they get for free.

  • Leon

    •

    Jul 20, 2024

    API Endpoint: The API endpoint for the Mixtral 8x7b model is:

    https://api.groq.com/openai/v1/completions

    Required Headers for Authentication: You'll need to provide an API key in the Authorization header. You can obtain an API key by signing up for a Groq account.

    Here's an example of the Authorization header:

    Authorization: Bearer YOUR_API_KEY

    Replace YOUR_API_KEY with your actual API key.

    Expected Request Format: The request body should be a JSON object with the following format:

    { 
  "prompt": "YOUR_PROMPT_HERE", 
  "model": "mixtral-8x7b", 
  "max_tokens": 1024, 
  "temperature": 0.5 
}

    Replace YOUR_PROMPT_HERE with the input text you want to process.

    Response Format: The response will be a JSON object with the following format:

    { 
  "choices": [ 
    { 
      "text": "GENERATED_TEXT_HERE", 
      "index": 0, 
      "logprobs": null 
    } 
  ] 
}

    The text field contains the generated text.

    Rate Limiting and Other Considerations:

    • Rate Limiting: Groq has a rate limit of 20 requests per minute for the free tier. You can check the rate limit status by sending a GET request to https://api.groq.com/openai/v1/rate_limit.

    • Maximum Input Length: The maximum input length is 2048 characters.

    • Supported Models: You can use other models like llama3-70b and gemma-7b-it by specifying them in the model field of the request body.