com.doji.transformers

    A Unity package to run pretrained transformer models with Unity Sentis

    Language: csharp

    Author: Nicolas Park (@nicolasp)

    398 stars · 1188 views

    Files

    • Vocab.cs (cs)
    • ClipTokenizer.cs (cs)
    • GPT2 (txt)
    • GPT2Tokenizer.cs (cs)
    • Auto (txt)
    • AutoModelForCausalLM.cs (cs)
    • AutoTokenizer.cs (cs)
    • Unity (txt)
    • TokenizerModelAsset.cs (cs)
    • Generation (txt)
    • Utils (txt)
    • GenerationMixin.cs (cs)
    • GenerateDecoderOnlyOutput.cs (cs)
    • ConfigurationUtils (txt)
    • GenerationConfig.cs (cs)
    • LogitsProcess (txt)
    • LogitsProcessor.cs (cs)
    • LogitsProcessorList.cs (cs)
    • StoppingCriteria (txt)
    • StopStringCriteria.cs (cs)
    • EosTokenCriteria.cs (cs)
    • StoppingCriteria.cs (cs)
    • MaxNewTokensCriteria.cs (cs)
    • MaxTimeCriteria.cs (cs)
    • MaxLengthCriteria.cs (cs)
    • StoppingCriteriaList.cs (cs)
    • BeamSearch (txt)
    • ConstrainedBeamSearchScorer.cs (cs)
    • BeamSearchScorer.cs (cs)
    • BeamConstraints.cs (cs)
    • TokenizationUtils (txt)
    • Token.cs (cs)
    • TokenizationUtils.cs (cs)
    • Trie.cs (cs)
    • PreTrainedTokenizer.cs (cs)
    • TokenString.cs (cs)
    • TokenizationUtilsBase (txt)
    • SpecialTokensMixin.cs (cs)
    • BatchEncoding.cs (cs)
    • InputEncoding.cs (cs)
    • PreTrainedTokenizerBase.cs (cs)
    • Input.cs (cs)
    • EncodingParams.cs (cs)
    • TokenizerConfig.cs (cs)
    • Kwargs.cs (cs)
    • ModelOutput.cs (cs)
    • Debug.cs (cs)
    • OrderedDictionary.cs (cs)
    • Log.cs (cs)
    • ConfigurationUtils (txt)
    • PretrainedConfig.cs (cs)
    • Configurable.cs (cs)
    • PretrainedModel.cs (cs)
    • PathUtils.cs (cs)
    • Models (txt)
    • Phi3 (txt)
    • Phi3ForCausalLM.cs (cs)
    • Llama (txt)
    • LlamaTokenizer.cs (cs)
    • Clip (txt)
    • BasicTokenizer.cs (cs)
    • LICENSE (txt)
    • .github (github)
    • FUNDING.yml (yml)
    • README.md (md)
    • projects (txt)
    • Transformers (txt)
    • Packages (txt)
    • packages-lock.json (json)
    • manifest.json (json)
    • ProjectSettings (txt)
    • ProjectVersion.txt (txt)
    • com.doji.transformers (transformers)
    • Editor (txt)
    • TokenizerModelAssetEditor.cs (cs)
    • TokenizerModelmporter.cs (cs)
    • Samples~ (txt)
    • 01-Phi3ForCausalLMSample (txt)
    • Phi3ForCausalLMSample.cs (cs)
    • Third Party Notices.md (md)
    • package.json (json)
    • readme.md (md)
    • Tests (txt)
    • Editor (txt)
    • LLamaTokenizerTest.cs (cs)
    • InputSerializationTest.cs (cs)
    • Resources (txt)
    • Phi-3-mini-4k-instruct (txt)
    • TrieTest.cs (cs)
    • BytesToUnicodeTest.cs (cs)
    • ClipTokenizerTest.cs (cs)
    • TokenizerConfigTest.cs (cs)
    • BasicTokenizerTest.cs (cs)
    • Runtime (txt)
    • AssemblyInfo.cs (cs)
    • Scripts (txt)
    • CacheUtils (txt)
    • Cache.cs (cs)
    • SinkCache.cs (cs)
    • StaticCache.cs (cs)
    • QuantizedCache.cs (cs)
    • HybridCache.cs (cs)
    • QuantoQuantizedCache.cs (cs)
    • DynamicCache.cs (cs)
    • EncoderDecoderCache.cs (cs)
    • QuantizedCacheConfig.cs (cs)
    • SlidingWindowCache.cs (cs)
    • HQQQuantizedCache.cs (cs)
    • OffloadedCache.cs (cs)
    • MambaCache.cs (cs)
    • Utils (txt)
    • ISpecialTokensMixin.cs (cs)
    • InputConverter.cs (cs)
    • AddedToken.cs (cs)
    • Encoding.cs (cs)
    • CHANGELOG.md (md)
    • ATTRIBUTION.md (markdown)

    Loading code snippet…