shipfeedAI news, curated daily

00:38:58 CET
21 MAY00:38:58shipfeed
pull to refreshlast sync
Just in — 30 new
§ feed · storyline

1/6-7/2024: LlaMA Pro - an alternative to PEFT/RAG??

LLaMA Pro research introduces an 8.3B parameter model expanding LLaMA2-7B with additional transformer layers trained on 80 billion tokens of code and math data to reduce catastrophic forgetting.

Jan 8 · · primary fetch1 sourceupdated Jan 8 ·

New research papers introduce promising Llama Extensions including TinyLlama, a compact 1.1B parameter model pretrained on about 1 trillion tokens for 3 epochs, and LLaMA Pro, an 8.3B parameter model expanding LLaMA2-7B with additional training on 80 billion tokens of code and math data. LLaMA Pro adds layers to avoid catastrophic forgetting and balances language and code tasks but faces scrutiny for not using newer models like Mistral or Qwen.

Meanwhile, OpenAI Discord discussions reveal insights on GPT-4 token limits, privacy reassurances, fine-tuning for GPT-3.5, challenges with multi-language image recognition, custom GPT creation requiring ChatGPT Plus, and security concerns in GPT deployment. Users also share tips on dynamic image generation with DALL-E and logo creation.

read full article on news.smol.ai
§ sources1 publication · timeline below
  1. news.smol.ai1/6-7/2024: LlaMA Pro - an alternative to PEFT/RAG??primary