Fix segment evolution keys

[glenn-jocher]

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Refinement of mutation results reporting in training scripts.

📊 Key Changes

• Changed the argument structure of the print_mutation function to accept keys as a parameter.
• Updated train.py and segment/train.py to specify keys when calling print_mutation.
• Reduced redundancy by removing the hardcoded keys list from the print_mutation function in utils/general.py.

🎯 Purpose & Impact

• 🛠️ Purpose: Streamlines the process of recording evolution data during hyperparameter tuning by making the desired keys for reporting more explicit and adaptable.
• 📈 Impact: Enhances the flexibility and maintainability of the code, which can lead to easier modifications and clearer mutation results for users engaging in hyperparameter optimization.

[AdrianStkr]

Hi,
thanks for your fast response. However, the bug is unfortunately still happening in the updated version.

python segments/train.py --data coco128-seg.yaml --weights yolov5s-seg.pt --img 640 --epochs 1 --batch-size 3 --evolve

still produces this:

File ".../PycharmProjects/yolo/segment/train.py", line 674, in
main(opt)
File ".../PycharmProjects/yolo/segment/train.py", line 654, in main
print_mutation(KEYS, results, hyp.copy(), save_dir, opt.bucket)
File ".../PycharmProjects/yolo/utils/general.py", line 977, in print_mutation
f.write(s + ('%20.5g,' * n % vals).rstrip(',') + '\n')
TypeError: not enough arguments for format string

Cheers,
Adrian

[glenn-jocher]

@AdrianStkr hi, thank you for your feature suggestion on how to improve YOLOv5 🚀!

The fastest and easiest way to incorporate your ideas into the official codebase is to submit a Pull Request (PR) implementing your idea, and if applicable providing before and after profiling/inference/training results to help us understand the improvement your feature provides. This allows us to directly see the changes in the code and to understand how they affect workflows and performance.

Please see our ✅ Contributing Guide to get started.

[AdrianStkr]

Hi,
I´d like to help you with a Pull Request implementing a fix to the bug. However, in order to do that I need some further info, which I have difficulties finding out myself.

• Can you specify the four different losses returned by segment/val.run? Can you further specify in which order the losses are returned?
• Is it feasible to change the order of keys in utils/segment/metrics.py without messing up other scripts referring to the list of keys in .../metrics.py?

best wishes,
Adrian

[glenn-jocher]

@AdrianStkr you can see the segmentation losses here:

yolov5/utils/segment/loss.py

Lines 104 to 111 in 16f87bb

	lbox *= self.hyp["box"]
	lobj *= self.hyp["obj"]
	lcls *= self.hyp["cls"]
	lseg *= self.hyp["box"] / bs
	loss = lbox + lobj + lcls + lseg
	return loss * bs, torch.cat((lbox, lseg, lobj, lcls)).detach()

I'd recommend not modifying the order of the keys in utils/segment/metrics.py

[AdrianStkr]

@glenn-jocher Thank you for the very fast response and the valuable information!

the correct set and order of keys used in print_mutation is:
keys = ('metrics/bbox_precision', 'metrics/bbox_recall', 'metrics/bbox_mAP_0.5', 'metrics/bbox_mAP_0.5:0.95', 'metrics/mask_precision', 'metrics/mask_recall', 'metrics/bbox_mAP_0.5', 'metrics/bbox_mAP_0.5:0.95', 'boxloss', 'objectloss', 'classloss', 'segmentloss')

I´ll implement the solution in a PR.
best wishes,
Adrian

[glenn-jocher]

@AdrianStkr great, thanks!

[ahmadtulsi]

I would like to ask question. I still get the same error after training when i enable --evolve parameter.

1 epochs completed in 0.002 hours.
Traceback (most recent call last):
File "segment/train.py", line 658, in
main(opt)
File "segment/train.py", line 638, in main
print_mutation(KEYS, results, hyp.copy(), save_dir, opt.bucket)
File "/content/yolov5/utils/general.py", line 1035, in print_mutation
f.write(s + ('%20.5g,' * n % vals).rstrip(',') + '\n')
TypeError: not enough arguments for format string

is there any solution for this?

[glenn-jocher]

@ahmadtulsi I've able to reproduce this with python segment/train.py --epochs 1 --evolve, but I don't have time to implement a fix. If you could work on a fix and submit a PR that would be very helpful. The area of the code for segmentation evolution is here:

yolov5/segment/train.py

Lines 556 to 645 in 443ef7f

	# Evolve hyperparameters (optional)
	else:
	# Hyperparameter evolution metadata (mutation scale 0-1, lower_limit, upper_limit)
	meta = {
	'lr0': (1, 1e-5, 1e-1), # initial learning rate (SGD=1E-2, Adam=1E-3)
	'lrf': (1, 0.01, 1.0), # final OneCycleLR learning rate (lr0 * lrf)
	'momentum': (0.3, 0.6, 0.98), # SGD momentum/Adam beta1
	'weight_decay': (1, 0.0, 0.001), # optimizer weight decay
	'warmup_epochs': (1, 0.0, 5.0), # warmup epochs (fractions ok)
	'warmup_momentum': (1, 0.0, 0.95), # warmup initial momentum
	'warmup_bias_lr': (1, 0.0, 0.2), # warmup initial bias lr
	'box': (1, 0.02, 0.2), # box loss gain
	'cls': (1, 0.2, 4.0), # cls loss gain
	'cls_pw': (1, 0.5, 2.0), # cls BCELoss positive_weight
	'obj': (1, 0.2, 4.0), # obj loss gain (scale with pixels)
	'obj_pw': (1, 0.5, 2.0), # obj BCELoss positive_weight
	'iou_t': (0, 0.1, 0.7), # IoU training threshold
	'anchor_t': (1, 2.0, 8.0), # anchor-multiple threshold
	'anchors': (2, 2.0, 10.0), # anchors per output grid (0 to ignore)
	'fl_gamma': (0, 0.0, 2.0), # focal loss gamma (efficientDet default gamma=1.5)
	'hsv_h': (1, 0.0, 0.1), # image HSV-Hue augmentation (fraction)
	'hsv_s': (1, 0.0, 0.9), # image HSV-Saturation augmentation (fraction)
	'hsv_v': (1, 0.0, 0.9), # image HSV-Value augmentation (fraction)
	'degrees': (1, 0.0, 45.0), # image rotation (+/- deg)
	'translate': (1, 0.0, 0.9), # image translation (+/- fraction)
	'scale': (1, 0.0, 0.9), # image scale (+/- gain)
	'shear': (1, 0.0, 10.0), # image shear (+/- deg)
	'perspective': (0, 0.0, 0.001), # image perspective (+/- fraction), range 0-0.001
	'flipud': (1, 0.0, 1.0), # image flip up-down (probability)
	'fliplr': (0, 0.0, 1.0), # image flip left-right (probability)
	'mosaic': (1, 0.0, 1.0), # image mixup (probability)
	'mixup': (1, 0.0, 1.0), # image mixup (probability)
	'copy_paste': (1, 0.0, 1.0)} # segment copy-paste (probability)
	with open(opt.hyp, errors='ignore') as f:
	hyp = yaml.safe_load(f) # load hyps dict
	if 'anchors' not in hyp: # anchors commented in hyp.yaml
	hyp['anchors'] = 3
	if opt.noautoanchor:
	del hyp['anchors'], meta['anchors']
	opt.noval, opt.nosave, save_dir = True, True, Path(opt.save_dir) # only val/save final epoch
	# ei = [isinstance(x, (int, float)) for x in hyp.values()] # evolvable indices
	evolve_yaml, evolve_csv = save_dir / 'hyp_evolve.yaml', save_dir / 'evolve.csv'
	if opt.bucket:
	os.system(f'gsutil cp gs://{opt.bucket}/evolve.csv {evolve_csv}') # download evolve.csv if exists
	for _ in range(opt.evolve): # generations to evolve
	if evolve_csv.exists(): # if evolve.csv exists: select best hyps and mutate
	# Select parent(s)
	parent = 'single' # parent selection method: 'single' or 'weighted'
	x = np.loadtxt(evolve_csv, ndmin=2, delimiter=',', skiprows=1)
	n = min(5, len(x)) # number of previous results to consider
	x = x[np.argsort(-fitness(x))][:n] # top n mutations
	w = fitness(x) - fitness(x).min() + 1E-6 # weights (sum > 0)
	if parent == 'single' or len(x) == 1:
	# x = x[random.randint(0, n - 1)] # random selection
	x = x[random.choices(range(n), weights=w)[0]] # weighted selection
	elif parent == 'weighted':
	x = (x * w.reshape(n, 1)).sum(0) / w.sum() # weighted combination
	# Mutate
	mp, s = 0.8, 0.2 # mutation probability, sigma
	npr = np.random
	npr.seed(int(time.time()))
	g = np.array([meta[k][0] for k in hyp.keys()]) # gains 0-1
	ng = len(meta)
	v = np.ones(ng)
	while all(v == 1): # mutate until a change occurs (prevent duplicates)
	v = (g * (npr.random(ng) < mp) * npr.randn(ng) * npr.random() * s + 1).clip(0.3, 3.0)
	for i, k in enumerate(hyp.keys()): # plt.hist(v.ravel(), 300)
	hyp[k] = float(x[i + 7] * v[i]) # mutate
	# Constrain to limits
	for k, v in meta.items():
	hyp[k] = max(hyp[k], v[1]) # lower limit
	hyp[k] = min(hyp[k], v[2]) # upper limit
	hyp[k] = round(hyp[k], 5) # significant digits
	# Train mutation
	results = train(hyp.copy(), opt, device, callbacks)
	callbacks = Callbacks()
	# Write mutation results
	print_mutation(KEYS, results, hyp.copy(), save_dir, opt.bucket)
	# Plot results
	plot_evolve(evolve_csv)
	LOGGER.info(f'Hyperparameter evolution finished {opt.evolve} generations\n'
	f"Results saved to {colorstr('bold', save_dir)}\n"
	f'Usage example: $ python train.py --hyp {evolve_yaml}')

[iker-lluvia]

I have created a PR (#11109 ) that fixes this error.

[glenn-jocher]

@iker-lluvia That's fantastic! Thank you for your contribution. I'll review the PR as soon as possible. Your help is much appreciated!