View mode

Cosmian AI¶

Cosmian AI aims to perform query over an AI runner (summary, translation, query, RAG) in a secured workflow.

Deploy Cosmian VM AI¶

Follow the instructions from the deployment guide.

Hardware optimization¶

When running on Intel Xeon processors, the application can leverage AMX (Advanced Matrix Extensions) to significantly enhance performance, especially for matrix-intensive operations commonly found in machine learning workloads. To enable this feature, simply start the application with the “use_amx” key to true from your config file.

  {
    "use_amx": true,       // Optional - default set to false
  }

HuggingFace authentication¶

To use certain Hugging Face models—such as the one required by the Translation endpoint—you must provide a Hugging Face access token. This token should be configured in the application’s configuration file.

  {
      "hf_token": "hf_token", // Mandatory (used for translation models)
  }

Usage¶

Cosmian AI Runner exposes endpoints which can be fetched:

Summarize text¶

Summary is made using “facebook/bart-large-cnn” model.

Endpoint: /summarize
Method: POST
Description: get the summary of a given text, using the configured model
Request:
Headers: ‘Content-Type: multipart/form-data’
Body: doc - text to summarize, using model configured on summary config section

Response:

  {
    "summary": "summarized text..."
  }

Example:

curl 'http://0.0.0.0:5000/summarize' \
--form 'doc="Il était une fois, dans un royaume couvert de vert émeraude et voilé dans les secrets murmurants des arbres anciens, vivait une princesse nommée Elara.."'

Translate text¶

Translation is made using one of the “Helsinki-NLP/opus-mt” models.

Endpoint: /translate
Method: POST
Description: get the translation of a given text, using model configured on translation config section
Request:
Headers: ‘Content-Type: multipart/form-data’
Body: doc - text to translate src_lang - source language tgt_lang - targeted language

Response:

  {
    "translation": "translated text..."
  }

Example:

curl 'http://0.0.0.0:5000/translate' \
--form 'doc="Il était une fois, dans un royaume couvert de vert émeraude et voilé dans les secrets murmurants des arbres anciens, vivait une princesse nommée Elara.."' --form 'src_lang=fr'  --form 'tgt_lang=en'

Predict using text as context¶

Endpoint: /context_predict
Method: POST
Description: get prediction from a model using current text as a context
Request:
Headers: ‘Content-Type: multipart/form-data’
Body: context - text to use as context for prediction query - query to answer

Example:

curl 'http://0.0.0.0:5000/context_predict' \
--form 'query="Who is Elara?"' --form 'context="Elara is a girl living in a forest..."'

Response: The response contains the answer to the query, from given context.

  {
    "result": ["Elara is the sovereign of the mystical forests of Eldoria"]
  }

Predict using RAG¶

Endpoint: /rag_predict
Method: POST
Description: get prediction from a model using RAG and configured documentary basis
Request:
Headers: ‘Content-Type: multipart/form-data’
Body: db - documentary basis to use for prediction query - query to answer

Example:

curl 'http://0.0.0.0:5000/rag_predict' \
--form 'query="Who is Esmeralda?"' --form 'db="literature"'

Response: The response contains the answer to the query, from given context.
```
  {
    "result": ["a street dancer"]
  }
```

You can list available documentary basis and their uploaded references from current configuration using:

Endpoint: /documentary_bases
Method: GET

Example:

curl 'http://0.0.0.0:5000/documentary_bases'

Response:

{
    "documentary_bases": {
        "literature": [
            "NDame de Paris"
        ],
        "science": []
    }
}

Manage references¶

You can add an .epub document, .docx document or a PDF to the vector DB of the given RAG associated to a database, using:

Endpoint: /add_reference
Method: POST
Request:
File sent on multipart
Body: db - database to insert reference reference - reference to insert

Example:

curl -F "file=@/data/doc.pdf" --form 'db="literature"' --form 'reference="crypto"'  http://0.0.0.0:5000/add_reference

Response:
```
File successfully processed
```

So far, only epub, pdf and docx files can be handled.

You can remove a reference to the vector DB of the given RAG associated to a database, using:

Endpoint: /delete_reference
Method: DELETE
Request:
Body: db - database to remove reference from reference - reference to delete

Example:

curl --form 'db="Literature"' --form 'reference="NDame de Paris"'  http://0.0.0.0:5000/delete_reference

Response:
```
Reference successfully removed
```

Authentication¶

If your AI Runner is configured to handle authentication, in order to make requests to Cosmian VM AI, users must be authentificated using your Identity Provider application (setup previously for Client-Side encryption). It should be a Single Page or Web application type.