Multi-task Learning Methods

Exciting breakthrough in making RAG systems more efficient and domain-specific! ServiceNow researchers have developed a novel approach to multi-task retriever fine-tuning that significantly improves retrieval performance across diverse enterprise use cases. >> Key Innovations Multi-Task Architecture The researchers fine-tuned mGTE-base (305M parameters) on multiple retrieval tasks simultaneously, including step retrieval, table retrieval, and field retrieval. Their solution achieved impressive recall scores of 0.90 for both step and table retrieval tasks, substantially outperforming baseline models like BM25 and other embedding models. Technical Implementation The system leverages instruction fine-tuning with a contrastive loss objective and employs sophisticated negative sampling strategies. The training process includes 5,000 steps with a batch size of 32, using gradient checkpointing and the Adafactor optimizer for memory efficiency. Cross-Domain Capabilities What's particularly impressive is the model's ability to generalize across different domains - from IT to HR and finance. The researchers validated this by testing on 10 different out-of-domain splits, demonstrating robust performance even in previously unseen contexts. Multilingual Support The fine-tuned model maintains strong multilingual capabilities, showing impressive results across German, Spanish, French, Japanese, and Hebrew translations of the development dataset. This work represents a significant step forward in making RAG systems more practical and efficient for enterprise applications. The researchers' approach not only improves retrieval accuracy but also addresses crucial real-world concerns like scalability and deployment costs.

Which approach works better for heart rate estimation? • A deep learning model trained specifically to extract heart rate from PPG (photoplethysmogram) signals? • Or a deep learning model trained to extract both heart rate and respiration rate concurrently from the same dataset? Our study shows that the second approach performs better. Interestingly, learning from one parameter extraction can actually improve the model’s performance in extracting the other. In our paper recently published in Computers in Biology and Medicine (Elsevier), we developed multi-task learning approaches that leverage shared characteristics across PPG-related tasks to enhance the performance of PPG applications. Find the paper here: https://lnkd.in/gX5J-5Ma Why does this matter? PPG is a rich signal containing information from multiple vital signs. Leveraging this information can help deep learning models perform better for a specific task. From a signal processing perspective, the information stored in the respiration rate range might not overlap with the heart rate information. However, the shared characteristics can still help deep learning models improve performance. Our contribution in this paper: We develop MLT models for two PPG applications: 1) Heart rate and respiration rate extraction 2) Heart rate and heart rate variability quality assessment The models were evaluated using a PPG dataset collected from 46 subjects during their daily routines via smartwatches. Our results showed that the proposed multi-task learning methods outperform baseline single-task models: achieving higher accuracy in signal quality assessment tasks and lower error rates in both heart rate and respiration rate estimation. This research highlights the potential of multi-task learning in improving physiological signal analysis and how it contributes to wearable health technology. Special thanks to Mohammad Feli, Kianoosh Kazemi, Pasi Liljeberg, and Amir M. Rahmani #MultiTaskLearning #PPG #HeartRate #RespirationRate #SignalProcessing #DeepLearning

Transfer Learning vs. Fine-tuning vs. Multitask Learning vs. Federated Learning, explained visually👇 . . Most ML models are trained independently without any interaction with other models. But real-world ML uses many powerful learning techniques that rely on model interactions. The following animation summarizes four such well-adopted and must-know training methodologies: 1) Transfer learning Useful when: - The task of interest has less data. - But a related task has abundant data. This is how it works: - Train a neural network model (base model) on the related task. - Replace the last few layers on the base model with new layers. - Train the network on the task of interest, but don’t update the weights of the unreplaced layers. Training on a related task first allows the model to capture the core patterns of the task of interest. Next, it can adapt the last few layers to capture task-specific behavior. 2) Fine-tuning Update the weights of some or all layers of the pre-trained model to adapt it to the new task. The idea may appear similar to transfer learning. But here, the whole pretrained model is typically adjusted to the new data. 3) Multi-task learning (MTL) A model is trained to perform multiple related tasks simultaneously. Architecture-wise, the model has: - A shared network - And task-specific branches The rationale is to share knowledge across tasks to improve generalization. In fact, we can also save computing power with MTL: - Imagine training 2 independent models on related tasks. - Now compare it to having a network with shared layers and then task-specific branches. Option 2 will typically result in: - Better generalization across all tasks. - Less memory to store model weights. - Less resource usage during training. 4) Federated learning This is a decentralized approach to ML. Here, the training data remains on the user's device. So in a way, it's like sending the model to data instead of data to model. To preserve privacy, only model updates are gathered from devices and sent to the server. The keyboard of our smartphone is a great example of this. It uses FL to learn typing patterns. This happens without transmitting sensitive keystrokes to a central server. Note: Here, the model is trained on small devices. Thus, it MUST be lightweight yet useful. Model compression techniques are prevalent in such cases. I have linked a detailed guide in the comments. -- If you want to learn AI/ML engineering, I have put together a free PDF (530+ pages) with 150+ core DS/ML lessons. Get here: https://lnkd.in/gi6xKmDc -- 👉 Over to you: What are some other ML training methodologies that I have missed here?

New Paper 🔔 PWM: Policy Learning with Large World Models Joint work with Varun Giridhar Nicklas Hansen Animesh Garg World models are becoming larger and more capable but why are we still learning policies with traditional sample-inefficient RL? We’re thrilled to introduce PWM, a multi-task RL method which solves 80 tasks across different embodiments in <10m per task using first-order gradient optimization. Check out the paper, code and models at http://imgeorgiev.com/pwm Recently, Nicklas Hansen unveiled TD-MPC2, scaling world models to 317M parameters paired with online planning. Our new method, PWM, leverages these world models to learn policies through an actor-critic architecture and optimizing via first-order gradients, known for their efficiency. We evaluate on locomotion tasks, with up to 152 action dimensions, and show PWM outperforming PPO by 26%. Astonishingly, it outperforms SHAC, an MBRL method using ground-truth dynamics, indicating that world models create smoother optimization landscapes than hand-engineered sims. We also tested PWM on 30 and 80 multi-task settings from dm_control and MetaWorld. After training a single large multi-task world model, we extract policies in <10 min per task. We surpassed TD-MPC2 by 27% and 8% respectively, without the need for online planning! This means that PWM can scale to billion-parameter models more effectively. Incredibly, multi-task PWM almost matches the performance of single-task experts like DreamerV3 and SAC #reinforcementlearning #research #machinelearning #AI #worldmodel #robotics