Problems with prediction ratios in multi-class training

[zengweigit]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hello, I am using YOLOV5. When I train a custom model, the prediction ratio of no_helmet is 96 when I only have one category. After I add a wrong_glove category, the prediction ratio of no_helmet is only 90. The same dataset is used for both trainings. Can you give me some optimization suggestions?

Additional

No response

[github-actions]

👋 Hello @zengweigit, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@zengweigit hello,

Thank you for reaching out and providing details about your issue. To assist you effectively, we need a bit more information. Could you please provide a minimum reproducible code example? This will help us better understand the context and reproduce the issue on our end. You can refer to our guide on creating a minimum reproducible example here: Minimum Reproducible Example.

Additionally, please ensure that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updates can resolve unexpected issues.

Regarding your specific problem with the prediction ratios, here are a few optimization suggestions:

1. Class Imbalance: Ensure that your dataset is balanced across all classes. An imbalanced dataset can lead to biased predictions.
2. Hyperparameter Tuning: Experiment with different hyperparameters such as learning rate, batch size, and epochs. Sometimes, fine-tuning these can improve model performance.
3. Data Augmentation: Apply data augmentation techniques to increase the variability of your training data, which can help the model generalize better.
4. Validation Set: Make sure you have a separate validation set to monitor the performance of each class during training.

If you can share more details or the code snippet, we can provide more targeted advice. Thank you for your cooperation, and we look forward to helping you resolve this issue.

[zengweigit]

I used the code from the https://github.com/ultralytics/yolov5/tree/v7.0 branch

The training command I executed was
python -m torch.distributed.run --nproc_per_node 2 train.py --data data/new_data.yaml --cfg models/new_yolov5s.yaml --weights pretrained/yolov5s.pt --epochs 100 --batch-size 256 --workers 16 --device 0,1 --cache ram --hyp data/hyps/hyp.scratch-med.yaml

new_data.yaml

new_yolov5s.yaml

What does class imbalance mean? Does it mean that the number of annotations for my no_helmet class is different from the number of annotations for the wrong_glove class? Or does it mean something else?

[glenn-jocher]

Hello @zengweigit,

Thank you for providing the details of your setup and the training command you used. It’s great to see that you are using the code from the v7.0 branch.

To address your question about class imbalance: Yes, class imbalance refers to a situation where the number of annotations (or instances) for each class in your dataset is significantly different. For example, if you have many more annotations for the no_helmet class compared to the wrong_glove class, your model might become biased towards predicting the more frequent class.

Steps to Address Class Imbalance:

1. Data Augmentation: Increase the number of instances for the minority class using data augmentation techniques. This can help balance the dataset.
2. Resampling: You can either oversample the minority class or undersample the majority class to balance the dataset.
3. Class Weights: Adjust the loss function to give more importance to the minority class. This can be done by setting class weights.

Example of Setting Class Weights:

You can modify the hyp.scratch-med.yaml file to include class weights. Here’s an example:

# Add class weights to the hyperparameters
cls: 0.5  # Class loss gain (original value)
cls_pw: [1.0, 2.0]  # Class weights for each class, adjust as needed

Verify Latest Versions:

Please ensure that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updates can resolve unexpected issues. You can update your repository with:

git pull

Minimum Reproducible Example:

If the issue persists, could you please provide a minimum reproducible code example? This will help us better understand the context and reproduce the issue on our end. You can refer to our guide on creating a minimum reproducible example here: Minimum Reproducible Example.

Feel free to reach out if you have any more questions or need further assistance. We're here to help! 😊