DistillKit v0.1 by Arcee Labs: The Technical Paper
Read the DistillKit v0.1 by Arcee AI Technical Paper: our new open-source tool that's set to change how we create and distribute Small Language Models (SLMs).
Shamane Siri
How Do I Prep my Data to Train an LLM?
So you want to train a custom language model, and you do have the requisite large set of text data. But how do you know that the data is *really actually ready* for model training? Our researchers here at Arcee AI tell you what to look out for.
Introducing the Ultimate SEC LLM: Revolutionizing Financial Insights
We built Llama-3-SEC upon the powerful Meta-Llama-3-70B-Instruct model, with the goal of providing unparalleled insights and analysis capabilities for financial professionals, investors, researchers, and anyone working with SEC filings and related financial data.
Why Methods Like QLoRA Fall Short in Domain Knowledge Injection
Arcee's research shows that the Standard Continual Pre-Training (CPT) approach performs better than QLORA-based CPT.
Case Study: Innovating Domain Adaptation through Continual Pre-Training and Model Merging
We show how Arcee uses the most innovative Continual Pre-Training and Model Merging techniques to deliver high-quality domain-specific language models at a fraction of the cost of our competitors–using Medical and Patent data.