Case Study: Approaches Tried in RemoveBG

1. Training the InSPyReNet Model for Better Masks

We first tried training the InSPyReNet model with the expectation that it would generate more accurate masks.

• What happened: The masks were still not detailed enough, especially around hair and the small gaps between subjects.
• Why it failed: The model was not able to capture fine boundary details accurately, which made it unsuitable for cases where edge precision was important.
• How it was addressed: We moved away from relying on InSPyReNet and evaluated stronger background removal models for better base mask quality.

2. Edge-Based Approach to Detect the Subject Boundary and Remove Green Around It

We researched an approach where we detect the subject edges and specifically remove green pixels around those areas.

• What happened: While this removed some green spill, it also started changing green clothing into blue around the edges.
• Why it failed: The method was too aggressive near subject boundaries and could not reliably distinguish between actual green spill and genuine green-colored objects like clothes.
• How it was addressed: We stopped using edge-based correction as a standalone method and limited it to controlled refinement inside post-processing.

3. Looking into Segmentation Models for Better Hair Handling

Since hair was one of the main problem areas, we started exploring segmentation models that could isolate hair more accurately. As part of this, we also tried Rembg models to generate masks.

• What happened: The masks produced by Rembg were still not accurate enough for detailed hair segmentation.
• Why it failed: The output lacked the quality needed for precise processing around hair, so it did not solve the main issue.
• How it was addressed: We continued evaluating better mask-generation approaches and later shifted to BiRefNet for stronger foreground extraction.

4. Research on MediaPipe, Combined with Gaussian Blur

We then explored MediaPipe to check whether it could provide better segmentation. Since MediaPipe alone was not giving strong results, we also researched adding Gaussian blur into the pipeline.

• Why it failed: MediaPipe by itself was not producing sufficiently accurate segmentation, and Gaussian blur only softened transitions rather than solving the actual masking and despill problem.
• How it was addressed: MediaPipe was retained as a supporting component in the pipeline, while Gaussian blur was used only for smoothing and not as the core fix.

5. Removing Blue Tones Introduced After Green Removal

During the pipeline improvements, we noticed that after green removal, blue tones were appearing in hair and clothing. To address this, we added logic to remove unwanted blue tones from these areas.

• What happened: This helped in some cases, but it also started affecting the original color of hair and clothing.
• Why it failed: The correction step was removing not only the unwanted blue spill but also valid color information, which caused visible color distortion.
• How it was addressed: We added a more controlled blue-spill removal step after green despill as part of post-processing.

6. Misclassification of Green Caps as Hair in MediaPipe

Another issue we faced was that green caps were being classified as hair by MediaPipe.

• What happened: Because of this misclassification, the caps also went through the same color correction process as hair and started turning blue.
• Why it failed: The segmentation was not fine-grained enough to clearly separate hair from accessories like caps, which caused incorrect color processing.
• How it was addressed: We reduced dependency on MediaPipe for primary segmentation and used it only as an additional support layer in the pipeline.

7. Research on SCHP (Self-Correction for Human Parsing)

To solve the cap-vs-hair problem, we explored SCHP, a human parsing model that provides separate masks for different regions such as hair, caps, and hats.

• What happened: SCHP did help by giving separate masks for caps and hats.
• Why it failed: Even with better region separation, the despill step was still not accurate enough. The color correction across hair and cap regions was still incorrect, so the final output was not reliable.
• How it was addressed: We used the learnings from SCHP for understanding region-level separation, but did not adopt it as the final solution.

8. BiRefNet for Background Removal

We explored BiRefNet to improve foreground-background separation and overall mask quality.

• What happened: BiRefNet gave better background removal results compared to the previously tested approaches.
• Why earlier methods failed in comparison: Earlier models did not provide a strong enough base mask for reliable post-processing and edge refinement.
• How it was addressed: BiRefNet was adopted as the primary background removal model in the current pipeline.