DexEvolve: Evolutionary Optimization for Robust and Diverse Dexterous Grasp Synthesis

Abstract

Use the simulator as an optimizer, not a filter

Dexterous grasping is fundamental to robotics, yet data-driven grasp prediction relies heavily on large and diverse datasets that are costly to generate and typically limited to a narrow set of gripper morphologies. Analytical grasp synthesis can scale data collection, but the required simplifying assumptions often yield physically infeasible grasps. These grasps must be filtered in high-fidelity simulators, substantially reducing both the number of retained grasps and their diversity.

We propose a scalable generate-and-refine pipeline for synthesizing large-scale, diverse, and physically feasible grasps. Instead of using high-fidelity simulators solely for verification and filtering, we leverage them as an optimization stage that continuously improves grasp quality without discarding precomputed candidates. We initialize an evolutionary search with a seed set of analytically generated, potentially suboptimal grasps, and refine these proposals directly in Isaac Sim using an asynchronous, gradient-free evolutionary algorithm that improves stability while maintaining diversity. The refinement can further be guided by human preferences and domain-specific quality metrics without requiring differentiable objectives. We then distill the refined grasp distribution into a diffusion model for robust real-world deployment. Experiments on our newly introduced Handles dataset and a DexGraspNet subset show that the proposed method produces over 120 distinct stable grasps per object, a 1.7–6× improvement over unrefined analytical seeds.

Refined grasps across the Handles dataset: 8 random grasps per asset, simulated in parallel.

Refined grasps on the Objects (DexGraspNet subset) dataset with the XHand.

Supplementary Video

Real-world experiments

Hardware validation on a Franka Panda arm with a dexterous XHand: 20 handle-grasping trials (5 handle types × 4 grasp poses) and object pickup tasks. Sim-to-real transfer without fine-tuning, executed in real time.

Method

Generate-and-refine evolutionary grasp synthesis

DexEvolve turns high-fidelity simulation from a binary accept/reject filter into a black-box objective that is optimized directly: instead of discarding failed seed grasps, it iteratively repairs and improves them.

1

Analytical seeding

A diverse seed set of grasp candidates is generated with an analytical optimizer (GraspQP). The framework is agnostic to the initialization strategy: any grasp generator, even random sampling, can provide the initial pool.

2

Evolutionary refinement in Isaac Sim

An asynchronous, gradient-free evolutionary algorithm refines grasps through massively parallel rollouts with early rejection. Density-aware selection and archive-based novelty insertion suppress redundant grasp clusters and prevent mode collapse, while contact points and grasping commands are adaptively resampled for each offspring.

3

Diffusion distillation

The refined grasp distribution is distilled into a point-cloud-conditioned diffusion model that diffuses the full grasp configuration, including wrist orientation, with keypoint-consistency and penetration losses, enabling fast inference from partial, noisy observations.

Key idea: Treat the physics simulator as a directly optimizable, black-box fitness function. Because the search is gradient-free, it natively supports non-differentiable objectives: simulated grasp lifetime under disturbances, task-specific metrics, and even learned human-preference rewards.

Evolutionary sampling in Isaac Lab: fitness terms (E_lifetime, E_penetration) evaluated live as the population evolves.

Adaptive contact-point resampling: active contacts are recomputed for every offspring and turned into grasping commands via the contact Jacobian.

Grasps without and with human preference alignment — **Preference-guided refinement.** Without guidance (left), refinement produces physically stable but unconventional grasps. Adding a PointNet++ preference model trained on ~1,000 human pairwise annotations to the fitness (right) steers evolution toward natural, human-like grasps while maintaining physical stability.

Handles Dataset

90 simulation-ready IKEA handle assets

90 geometrically distinct door handle and knob assets with multiple texture variations, modeled after commercially available IKEA products, enabling reproducible evaluation and direct sim-to-real testing with physically purchasable hardware.

Overview of the IKEA Handles dataset assets — **Selection of the Handles dataset.** Each handle includes high-resolution collision geometry (re-meshed to ≤1 cm triangles for stable SDF contact simulation), realistic textures, and articulated joints with compliant degrees of freedom. Released as articulated `.usd` assets that work directly in Isaac Sim.

Results

More stable and more diverse

Evolutionary refinement consistently improves both stability and unique-grasp coverage across all initialization strategies and both datasets, improving distinct stable grasps (DSG) from 19 to 114 with 32 analytical seeds and from 69 to 118 with 128 seeds, while preserving or increasing entropy across position, orientation, and joint spaces.

Radar plots comparing grasp quality and diversity across generation strategies — **Grasp quality and diversity comparison.** Success rate, DSG@2cm/DSG@20cm, and marginal entropies of position, orientation, and joints across generation strategies on *Objects* and *Handles*. Hatched bars denote evolutionary refinement; non-hatched bars denote baselines. Refinement improves stability while preserving diversity across both datasets.

Cross-hand generalization

The framework is hand-agnostic: gains are consistent across parallel-jaw and fully dexterous hands on Objects, using 32-seed GraspQP initialization followed by evolutionary refinement.

Metric	Robotiq 2F	Robotiq 3F	Allegro	Shadow
DSG@5cm (seed → refined)	9 → 80	25 → 86	25 → 123	17 → 110
Entropy H (seed → refined)	1.5 → 2.7	2.3 → 2.7	2.4 → 2.9	2.1 → 2.8

Bold = after evolutionary refinement. DSG@5cm counts distinct stable grasps at 5 cm resolution; H is the average entropy over position, orientation, and joint states.

Convergence of distinct stable grasps and entropy over environment steps — **Convergence.** Distinct stable grasps and entropy improve rapidly during the first 2.5k simulator steps and plateau by ~10k steps; over 90% of final quality is reached in the first quarter of the budget, so early stopping can substantially reduce cost (≈2.4 min/object for 128 grasps).

Bar chart comparing analytical seeds, diffusion training, and evolutionary refinement — **Diffusion vs. evolutionary refinement.** A diffusion model trained on raw analytical seeds improves over its training data (e.g. 20 → 71 unique grasps at 32 seeds) but consistently underperforms simulator-in-the-loop refinement (115 at 32 seeds, 118 at 128 seeds) in both stability and entropy. The two components are complementary: refinement for offline data quality, diffusion for distillation and partial-observation inference.

Real-World Deployment

Sim-to-real without fine-tuning

A Franka Panda arm with a dexterous XHand grasps handles, knobs, and household objects: multi-view RGB-D perception with DepthAnything-v3 depth alignment, grasp sampling from the refined diffusion model with collision-aware guidance, and cuRobo motion planning.

Real-world cabinet handle grasping rollouts with a Franka arm and XHand — **Real-world cabinet-handle rollouts.** Handle grasps are sampled from a point-cloud-conditioned diffusion model trained on refined grasp distributions, planned with cuRobo, and executed with a joint impedance controller. Three executions across different grasp modes and approach directions demonstrate successful sim-to-real transfer in contact-rich manipulation.

Diffusion denoising on a real point cloud: random noise → grasp prediction over 100 steps.

Handle grasping across 5 handle types × 4 grasp poses, executed in real time.

Household object pickup with grasps distilled from the refined distribution.

Citation

BibTeX

@article{zurbruegg2026dexevolve,
  title   = {DexEvolve: Evolutionary Optimization for Robust and Diverse Dexterous Grasp Synthesis},
  author  = {Zurbr{\"u}gg, Ren{\'e} and Cramariuc, Andrei and Hutter, Marco},
  journal = {arXiv preprint arXiv:2602.15201},
  year    = {2026}
}