Insights

Optimizing LLM Training with Spectrum

Optimizing LLM Training with Spectrum

Here at Arcee AI, we're the pioneers of training performant and efficient LLMs with Model Merging... And now we bring you *yet another* cutting-edge technique that also dramatically optimizes your training and improves your models.

Evolutionary Model Merging For All

Evolutionary Model Merging For All

We've been focused on developing this groundbreaking technique for the community, and we're now excited to announce the launch of this state-of-the-art functionality in MergeKit.

Why Methods Like QLoRA Fall Short in Domain Knowledge Injection

Why Methods Like QLoRA Fall Short in Domain Knowledge Injection

Arcee's research shows that the Standard Continual Pre-Training (CPT) approach performs better than QLORA-based CPT.

How to Merge Llama3 Using MergeKit

How to Merge Llama3 Using MergeKit

... And what do we do at Arcee when an exciting new model drops? We MERGE IT on MergeKit! We walk you through the process and share the initial results.

Case Study: Innovating Domain Adaptation through Continual Pre-Training and Model Merging

Case Study: Innovating Domain Adaptation through Continual Pre-Training and Model Merging

We show how Arcee uses the most innovative Continual Pre-Training and Model Merging techniques to deliver high-quality domain-specific language models at a fraction of the cost of our competitors–using Medical and Patent data.

What are the top 5 industries ripe for SLM adoption?

What are the top 5 industries ripe for SLM adoption?

LLMs are so 2023, and SLMs (Small Language Models) are the future of enterprise GenAI. Our Solutions Engineering Lead Tyler Odenthal wrote about how domain-adapted SLMs are being used across diverse sectors ranging from law and healthcare to education and customer service.

When should I use LLMs vs SLMs?

When should I use LLMs vs SLMs?

When it comes to the world of language models and Gen AI, a key question for companies looking to adopt these innovations is which model(s) to use. As if it’s not already complicated enough with the plethora of foundational models out there, it is now even more daunting