Library

We offer the Maritalk Python library to facilitate integration with our API. We recommend using the version compatible with OpenAI (detailed in OpenAI Compatibility), which is ideal for those already using OpenAI libraries or seeking compatibility with this platform. For users who previously utilized our standalone library version, we maintain this option to ensure continuity for existing projects. Both versions share the same core functionalities, allowing you to choose the one that best fits your workflow.

Installing the Maritalk Python Library

With Python installed, and optionally with a virtual environment activated and pip updated, you can install the Maritalk library. In the terminal/command line, run:

pip install maritalk

With the virtual environment activated, you can list all installed Python libraries within this environment using the command pip list. Open the terminal or command prompt and type:

pip list

If the installation was successful, you’ll see something similar to:

maritalk X.X.X

where X.X.X is the version number of the Maritalk library you installed.

Sending a Request to the API

After setting up Python and configuring an API key, you can send a request to the Maritalk API using the Python library. To do this, create a file called maritalk.py using the terminal or an IDE. Inside the file, copy and paste one of the examples below:

import maritalk

model = maritalk.MariTalk(
    key="insert your key here. Ex: '100088...'",
    model="sabiazinho-4"  # The list of supported models is available at https://docs.maritaca.ai/en/models
)

response = model.generate("What is 25 + 27?", max_tokens=8000)
answer = response["answer"]

print(f"Answer: {answer}")   # Should print something like "25 + 27 equals 52."

Note that the response dictionary contains the usage key, which shows the number of input and output tokens that will be billed. To run the code, type python maritalk.py in the terminal/command line.

Streaming

For long text generation tasks, such as creating an extensive article or translating a large document, it may be advantageous to receive the response in parts as the text is generated rather than waiting for the entire text. This approach makes the application more responsive and efficient, especially when the generated text is extensive. We offer two approaches to meet this need: using a generator and an async_generator.

Generator

• By using stream=True, the code will return a generator. This generator will provide parts of the response as they are generated by the model, allowing you to print or process the tokens as they are produced.

for response in model.generate(
    messages,
    do_sample=True,
    max_tokens=200,
    temperature=0.7,
    top_p=0.95,
    stream=True,
    num_tokens_per_message=4
):
    print(response)

AsyncGenerator

When using stream=True in conjunction with return_async_generator=True, the code will return an AsyncGenerator. This type of generator is designed to be consumed asynchronously, allowing you to run the code that consumes the AsyncGenerator concurrently with other tasks, improving processing efficiency.

import asyncio

async_generator = model.generate(
    messages,
    do_sample=True,
    max_tokens=200,
    temperature=0.7,
    top_p=0.95,
    stream=True,
    return_async_generator=True,
    num_tokens_per_message=4
)

async def consume_generator():
    async for response in async_generator:
        print(response)
                # Your code here...

asyncio.run(consume_generator)

Chat Mode

You can set up a conversation by specifying a list of dictionaries, with each dictionary containing two keys: content and role.

Currently, the Maritaca API supports three values for role: "system" for chatbot instruction messages, "user" for user messages, and "assistant" for assistant messages.

An example conversation is shown below:

messages = [
    {"role": "user", "content": "suggest three names for my dog"},
    {"role": "assistant", "content": "Nina, Bella, and Luna."},
    {"role": "user", "content": "and for my fish?"},
]

answer = model.generate(
    messages,
    do_sample=True,
    max_tokens=200,
    temperature=0.7,
    top_p=0.95)["answer"]

print(f"Answer: {answer}")   # Should print something like "Nemo, Dory, and Neptune."

Few-Shot Examples

While Sabia can respond to instructions without any demonstration examples, providing a few task examples can significantly improve the quality of its responses.

Below, we show how this is done for a simple sentiment analysis task, i.e., classifying whether a movie review is positive or negative. In this case, we will pass two few-shot examples, one positive and one negative, and a third example, for which Sabia will actually make the prediction.

prompt = """Classify the movie review as "positive" or "negative".

Review: I really liked the movie, it's the best of the year!
Class: positive

Review: The movie leaves much to be desired.
Class: negative

Review: Although long, it was worth the ticket..
Class:"""

answer = model.generate(
    prompt,
    chat_mode=False,
    do_sample=False,
    max_tokens=20,
    stopping_tokens=["\n"]
)["answer"]

print(f"Answer: {answer.strip()}")  # Should print "positive"

Note that we use chat_mode=False as it improves the quality of responses when using few-shot examples.

The argument stopping_tokens=["\n"] is used to stop generation when the "\n" token is generated. This is necessary because, when not in chat mode, the model might not know when to stop generating.

For tasks with only one correct answer, like the example above, it is recommended to use do_sample=False. This ensures the same response is generated given a specific prompt.

For diverse or long text generation tasks, it is recommended to use do_sample=True and temperature=0.7. The higher the temperature, the more diverse the generated texts will be, but there is a greater chance the model may "hallucinate" and generate nonsensical text. The lower the temperature, the more conservative the response, but it risks generating repetitive text.