Pinned Loading

1. creating_Phi2_MoE_using_mergekit.md creating_Phi2_MoE_using_mergekit.md

   1

   # microsoft/phi-2 for creating Mixture of Experts (MoE)

   2

   The [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) is a small language model with 2.7billion parameters. Because of its small size, opensource license and thanks to finetuning techqniques like QLoRA, one can (fairly) quickly finetune a base model for performing downstream tasks and creating an expert phi-2 model. It would be interesting to combine the individual experts into a Mixture of Experts (MoE) to make the MoE perform the tasks of the individual experts. Follow the steps below to create your own version of a MoE based out of phi-2.

   3

   - Special mention to [Maxime Labonne](), [Aratako](https://github.com/Aratako), [Paul Ilioaica](https://github.com/paulilioaica) for showing the opensource community that the mergekit can be tweaked to make a MoE out of phi-2 experts.

   4

   - Big shoutout to [Charles O. Goddard](https://github.com/cg123), the author of mergekit for creating and letting us play with [mergekit](https://github.com/arcee-ai/mergekit)

   5

2. berts.cpp-on-android berts.cpp-on-android Public

   Forked from yilong2001/berts.cpp

   CPP inference of Bert-model family based on GGML, supporting sentence classification models, seq2seq text generation models, etc.

   C 1

3. SORT-java SORT-java Public

   A Java implementation of the SORT (Simple Online and Realtime Tracking) algorithm, ready to integrate into Android applications.

   Java

4. mergekit mergekit Public

   Forked from arcee-ai/mergekit

   Tools for merging pretrained large language models.

   Python 19 1

5. FREEkMapper FREEkMapper Public

   A powerful projection mapping application built with Python, Tkinter, and OpenGL. Designed for live performance and installation art, Freakmapper allows you to map video and images onto physical su…

   Python 1