Human Centered AI – Technical Exchange Meetings

Date: 11/06/2024

Vendor: Anote

Technology/Topic: Human Centered AI

______________________________________________

Welcome to the Technical Exchange Meeting (TEM)!

Anote is an end-to-end MLOps platform that enables you to obtain the best large language model for your data. Anote’s solution consists of 3 fully functional products:

1) Label Text Data – Classify Text, Extract Entities and Answer Questions on Documents

2) Fine Tune Model – Run your Fine Tuned LLM locally with our model inference API

3) Private Chatbot – Chat with your documents while keeping your data private and secure.

The core technology is a fine-tuning library that leverages state of the art few shot learning to make high quality predictions with a few labeled samples. There are 3 ways to do fine tuning:

1) Unsupervised Fine Tuning – Fine tune your LLM from your raw unstructured documents

2) Supervised Fine Tuning – Fine tune your LLM on your labeled data

3) RLHF / RLAIF Fine Tuning – Actively improve your models with human / AI feedback

Anote provides a data annotation interface to convert raw unstructured data into an LLM ready format and incorporate subject matter expertise into your training process to improve model accuracies. You can use the data annotation interface with 4 main steps:

1) Upload – Create a new text-based dataset

2) Customize – Add the categories, entities, or questions you care about

3) Annotate – As you annotate a few edge cases, the LLM actively learns to predict the rest

4) Download – Download the resulting CSV / export the fine-tuned LLM as an API endpoint

End users can route / integrate the best LLM into their own, on premise, private chatbot, as well as interact with our software development kit. To use the Private Chatbot, there are 3 steps.

1) Upload – Upload your documents

2) Chat – Ask questions on your documents with LLMs like GPT, Claude, Llama2 and Mistral

3) Evaluate – Get citations for answers, and mitigate the effect of hallucinations

Anote provides an evaluation framework to compare zero shot LLMs like GPT, Claude, Llama3 and Mistral, with fine-tuned LLMs that are trained on your domain specific training data. Our evaluation dashboard enables you to see the improved model performance of supervised and unsupervised fine-tuning models with metrics like Cosine Similarity, Rouge-L Score, LLM Eval, Answer Relevance and Faithfulness.

______________________________________________

Upcoming TEM events can be viewed at the following site: https://zadar.us/tem/schedule.html

______________________________________________

Disclaimer:

— TEMs do not serve as a marketing venue or request for proposal actions.

— TEMs shall not be interpreted as a commitment by the Government to issue a solicitation or ultimately award a contract.

— TEMs do not serve as an endorsement of any presented technologies or capabilities

— Presentations will not be considered as proposals nor will any awards be made as a result of a TEM session.

— TEMs are public open forums – no proprietary or sensitive information should be presented during TEM sessions. Only publicly facing content is permissible in DISA TEM sessions.