shipfeedAI news, curated daily

23:04:11 CET
20 MAY23:04:11shipfeed
pull to refreshlast sync
Just in — 30 new
§ feed · storyline

1/17/2024: Help crowdsource function calling datasets

LM Studio clarifies its closed-source, free-for-personal-use status in an updated FAQ, while a new beta release introduces fixes and hints at forthcoming 2-bit quantization support.

Jan 18 · · primary fetch1 sourceupdated Jan 18 ·

LM Studio updated its FAQ clarifying its closed-source status and perpetual freeness for personal use with no data collection. The new beta release includes fixes and hints at upcoming 2-bit quantization support. For gaming, models like Dolphin 2.7 Mixtral 8x7B, MegaDolphin, and Dolphin 2.6 Mistral 7B DPO with Q4_K_M quantization were recommended. Discussions highlighted that single powerful GPUs outperform multi-GPU setups due to bottlenecks, with older GPUs like Tesla P40 being cost-effective.

Microsoft's AutoGen Studio was introduced but has issues and requires API fees for open-source models. Linux users are advised to use llama.cpp over LM Studio due to lack of headless mode. Additional tools like LLMFarm for iOS and various Hugging Face repositories were also mentioned. "LM Studio must be running to use the local inference server as there is no headless mode available" and "matching model size to GPU memory is key for performance" were notable points.

read full article on news.smol.ai
§ sources1 publication · timeline below
  1. news.smol.ai1/17/2024: Help crowdsource function calling datasetsprimary
1/17/2024: Help crowdsource function calling datasets · shipfeed