Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Safety Checker for FAL Image Generation #813

Closed
daojonesceo opened this issue Dec 3, 2024 · 1 comment
Closed

Disable Safety Checker for FAL Image Generation #813

daojonesceo opened this issue Dec 3, 2024 · 1 comment
Labels
enhancement New feature or request no-issue-activity

Comments

@daojonesceo
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Many image generation services such as FAL, Stable Diffusion, etc. have "safety checkers" that ensure prompts are appropriate.
These have several problems:

  1. Additional latency: These checks are done via an LLM, which can add unnecessary latency.
  2. Unpredictable / incorrect outputs: Since the checks use LLMs, their output can be overly restrictive or random, causing unpredictable behavior.
  3. Not necessary: Some use cases don't require the safety checker.

In particular, we at daojones.ai (@DAOJonesPumpAI on X.com) have found this is overly restrictive and have written a custom image generation application that disables it.

Describe the solution you'd like

We want to set the FAL configuration for our image requests.
We need to set enable_safety_checker to false while also setting safety_tolerance to 6 (the most tolerant setting).
Since these options are not uniformly supported by the different APIs, we will just add an "escape hatch" configuration imageSettings.fal.

Describe alternatives you've considered

  1. Turning safety checker off globally: This would work for our use case but could be a breaking change for other use cases.
  2. Adding a global allowNsfw flag: This requires reconciling this with other APIs and determining what settings it would correspond to.
  3. Adding the enableSafetyChecker / safetyTolerance flags at the top level of imageSettings: this would pollute the global namespace.

Additional context

@daojonesceo daojonesceo added the enhancement New feature or request label Dec 3, 2024
Copy link
Contributor

github-actions bot commented Jan 2, 2025

This issue has been automatically marked as stale due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request no-issue-activity
Projects
None yet
Development

No branches or pull requests

1 participant