v0.8.4

[justinxzhao]

What's Changed

• Add codellama to tokenizer list for set_pad_token by @Infernaught in Add codellama to tokenizer list for set_pad_token #3598
• Set default eval batch size to 2 for LLM fine-tuning by @arnavgarg1 in Set default eval batch size to 2 for LLM fine-tuning #3599
• [CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. by @justinxzhao in [CI] Explicitly set eval batch size in determinism tests, introduce a new integration test group, and exclude slow tests. #3590
• [CI] Run sudo apt-get update in GHAs. by @justinxzhao in [CI] Run sudo apt-get update in GHAs. #3608
• Store steps_per_epoch in Trainer by @hungcs in Store steps_per_epoch in Trainer #3601
• Updated characters, underscore and comma preprocessors to be TorchScriptable. by @martindavis in Updated characters, underscore and comma preprocessors to be TorchScriptable. #3602
• [CI] Deflake: Explicitly set eval batch size for mlflow test. by @justinxzhao in [CI] Deflake: Explicitly set eval batch size for mlflow test. #3612
• Fix registration for char error rate. by @justinxzhao in Fix registration for char error rate. #3604
• fix: Load 8-bit quantized models for eval after fine-tuning by @jeffkinnison in fix: Load 8-bit quantized models for eval after fine-tuning #3606
• Add Code Alpaca and Consumer Complaints Datasets by @connor-mccorm in Add Code Alpaca and Consumer Complaints Datasets #3611
• Add support for gradient checkpointing for LLM fine-tuning by @arnavgarg1 in Add support for gradient checkpointing for LLM fine-tuning #3613
• Bump min support transformers to 4.33.0 by @tgaddair in Bump min support transformers to 4.33.0 #3616
• [CI] Fix failing tests on master by @arnavgarg1 in [CI] Fix failing tests on master #3617
• Eliminate short-circuiting for loading from local by @Infernaught in Eliminate short-circuiting for loading from local #3600
• Refactor integration tests into matrix by @tgaddair in Refactor integration tests into matrix #3618
• fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval by @jeffkinnison in fix: Check underlying model device type when moving 8-bit quantized models to GPU at eval #3622
• Fixed range validation for text generation penalty parameters by @tgaddair in Fixed range validation for text generation penalty parameters #3623
• Update comment for predict to update Ludwig docs by @Infernaught in Update comment for predict to update Ludwig docs #3535
• Avoid deprecation warnings on pandas Series.fillna by @carlogrisetti in Avoid deprecation warnings on pandas Series.fillna #3631
• QoL: Default to using fast tokenizer for Llama models by @arnavgarg1 in QoL: Default to using fast tokenizer for Llama models #3625
• fixed typo in EfficientNet's model variant from v2_ to v2_s by @saad-palapa in fixed typo in EfficientNet's model variant from v2_ to v2_s #3628
• Add pytorch profiler and additional tensorboard logs for GPU memory usage. by @justinxzhao in Add pytorch profiler and additional tensorboard logs for GPU memory usage. #3607
• Pin minimum transformers version to 4.33.2 by @arnavgarg1 in Pin minimum transformers version to 4.33.2 #3637

New Contributors

• @saad-palapa made their first contribution in fixed typo in EfficientNet's model variant from v2_ to v2_s #3628

Full Changelog: v0.8.3...v0.8.4

---

This discussion was created from the release v0.8.4.