Mastering the Automatic1111 User Interface & WebUI Forge: A Comprehensive Guide for Stable Diffusion (2024)

What is Automatic 1111 web ui?

Automatic1111 Web UIis a browser-based interface forStable Diffusion, a powerful open-source AI art generator. It provides a user-friendly way to interact with Stable Diffusion models and create AI images. Here are some key features of theAutomatic1111 Web UI:

To install the Web UI, check out this guide below:

What is Webui Forge?

Stable Diffusion WebUI Forgeis a platform built on top ofStable Diffusion WebUI, which is based on theGradioframework.The name “Forge” draws inspiration from “Minecraft Forge.” This project aims to enhance the functionality and efficiency of the originalStable Diffusion WebUIby making development easier, optimizing resource management, and speeding up inference.

Key Features ofStable Diffusion WebUI Forge:

• Speed Improvements:

  • The GPU memory peak (as seen in the task manager) will decrease by about700MB to 1.3GB.
  • The maximum diffusion resolution (that won’t cause out-of-memory errors) will increase by approximately2x to 3x.
  • The maximum diffusion batch size (that won’t cause out-of-memory errors) will increase by about4x to 6x.
  • For less powerful GPUs (e.g., 6GB VRAM), the speed-up is even more significant.
  • Even powerful GPUs (e.g., 4090 with 24GB VRAM) benefit from some improvements, albeit to a lesser extent.

1. What is VAE?
   • VAEstands forVariational Autoencoder. It’s a neural network component that encodes and decodes images into and from a smaller latent space. This compression allows for faster computation.
   • In the context ofStable Diffusion, VAEs play a crucial role in improving image quality by recovering fine details.
2. Do I Need a VAE?
   • By default,Stable Diffusionmodels (whether v1, v2, or custom) come with a built-in VAE.
   • However, when people refer to downloading and using a VAE, they mean using animproved versionof it. These improved versions result from further fine-tuning the VAE part of the model with additional data.
   • Instead of releasing an entirely new model, only the updated VAE portion is provided as a smaller file.
3. Effect of Using VAE:
   • The impact of using VAE is usually subtle but significant.
   • An improved VAE enhances image decoding from the latent space, leading to better recovery of fine details.
   • It particularly benefits rendering features likeeyesandtext, where fine details matter.
4. Stability AI’s Fine-Tuned VAEs:
   • Stability AIhas released two variants of fine-tuned VAE decoders:
     • EMA (Exponential Moving Average): Produces sharper images.
     • MSE (Mean Square Error): Yields somewhat smoother outputs.
   • Both versions improve image quality without any degradation.
5. Comparison(256×256 images):
   • Original VAE: Used inStable Diffusion.
   • ft-EMA VAE: Slightly better overall.
   • ft-MSE VAE: Emphasizes MSE reconstruction.
6. Example Comparison(512×512 images):
   • Rendering eyes improves, especially for small faces.
   • Text rendering improvements are less pronounced.
   • Neither EMA nor MSE performs worse; they either do better or maintain the same quality.

1. Download the required file and place it in the stable-diffusion-webui/models/VAE/ directory.
2. Navigate to the Settings Tab on the A1111 Webui, select Stable Diffusion from the left-side menu, click on SD VAE, and then choose ‘vae-ft-mse-840000-ema-pruned’.
3. Hit the ‘Apply Settings‘ button and patiently wait for a successful application message.
4. Proceed to generate your image in the usual manner, using any Stable Diffusion model in either the txt2img or img2img options.

• To enable SD VAE, make sure you turn it on in the settings.
  • Goto Settings > User Interface > [info] Quicksettings list > add sd_vae

Generation Settings

Prompt Engineering

• Negative promptssteer the AI model away from certain outcomes.

  • Instead of specifying what you want, you indicate what you don’t want. These are essential for newer AI models (v2) to improve image quality and alignment with expectations.
  • For example, if you want to avoid certain attributes (e.g., “ugly,” “deformed”), use negative prompts to guide the model away from those aspects.

What are Tokens for?

• Tokens and the 0/75 Indicator

  In the context of Stable Diffusion, a ‘token’ is essentially a unit of text input you can feed to the model. The number “0/75” displayed at the far end of your prompt bar represents the standard maximum limit of tokens you can use in a single prompt. Anything beyond that may not be considered by the model.

• Tokens Beyond 75 – Infinite Prompt Length

  But what if you have more to say? That’s where ‘Infinite Prompt Length’ comes in. If you type more than 75 tokens, the model cleverly divides your prompt into smaller pieces, each consisting of 75 tokens or fewer, and processes each chunk independently. So, if you have a 120-token prompt, it gets divided into a 75-token chunk and a 45-token chunk, which are processed separately and then combined. This allows you to provide more complex instructions to the model.

• The ‘BREAK’ Keyword

  If you want to start a new chunk of tokens before reaching the 75-token limit, you can use the ‘BREAK’ keyword. This command will fill the remainder of the current chunk with ’empty’ tokens, and any text you type afterwards will start a new chunk. This gives you more control over how your input is processed. And there you have it! A simplified overview of tokens, infinite prompt length, and the BREAK keyword in Stable Diffusion.

Note: Each letter, number, and punctuation mark counts as a token, but spaces between them do not.

Prompt Style

ThePrompt Stylesection inAutomatic1111/Forgeprovides tools for managing and customizing prompt styles. Here’s a breakdown of its components:

• Buttons and Dropdown Menu:

  

  • Four buttonsand adropdown menuconstitute this section.
  • These elements allow you to create presets or upload CSV files containing attributes related to image styles.

• Button Functions:
  Click on the paintbrush icon to the right and the below menu will appear. Here, you will type in your information and save your presets. This becomes valuable as you start to collect data as you become more experienced to save time.

  

  • Blue Button (“Read generation parameters from prompt or last generation if prompt is empty”):
    • This button extracts generation parameters from the prompt or the last generated image (if the prompt is empty) and populates them in the user interface.
  • Trash Bin Icon (“Clears the prompts in the prompt and negative prompt box”):
    • Clicking this icon clears the content in the prompt and negative prompt boxes.
  • Paint Brush Icon (“Edit Style”):
    • Clicking this icon opens a menu where you can edit and save custom styles.
  • Dropdown Menu:
    • The dropdown menu allows you to select from available prompt styles that you can use for your creative process.

While these samplers are essential for Stable Diffusion, they are just one part of the entire model.If you want to learn more, I recommend reading my guide on Sampling Methods for Stable Diffusion. Below is a list of sampling methods available in both Automatic1111 and WebUI Forge.

• Upscaling During Generation:

  • Normally, you’d generate an image first and then upscale it usingimg2imgor other options in Automatic1111.
  • However,HiRes.fixstreamlines this process by increasing the resolutionduring image generationitself.
  • This approach ensures that no artifacts or irregularities are introduced during upscaling.
  • Additionally, it enhances the overall image quality.

• Setting the Target Resolution:

  • When you openHiRes.fix, you’ll notice that it’s initially set to‘Upscale by 2’.
  • This means that the default width and height (512×512) will be doubled, resulting in a1024×1024image.
  • You can further upscale by a maximum factor of4x.
  • Be aware that the more you upscale, the longer it will take to generate the image.

• Choosing an Upscaler:

  • HiRes.fixallows you to select from various upscalers available in a dropdown menu.
  • Each upscaler has unique characteristics, so experimentation is key.
  • Here are some options:
    • Latent Upscaler: A good general-purpose choice.
    • R-ESRGAN 4x+: Ideal for photorealistic images.
    • R-ESRGAN 4x+ Anime6B: Works well for animated/cartoon-style images.

• HiRes Steps: Post-Sampling Brilliance:

  • HiRes stepsrefine image quality after the initial sampling.
  • These steps occur after the sampling steps and contribute to the overall image quality.
  • You can setHiRes stepsin the range of0–150.
  • Keeping it at0makes HiRes steps equal to the sampling steps.
  • For example, if you have 20 sampling steps and 0 HiRes steps, the total steps would be 40.

• Width and Height Parameters:

  • In Stable Diffusion, theWidthandHeightparameters determine the size of the generated images.
  • By default, Stable Diffusion produces images with dimensions of512×512 pixels.
  • You can customize these dimensions to create rectangular images with specific aspect ratios.
  • Keep in mind that both the width and height should bemultiples of 8to avoid distortions.
  • Aspect ratios can differ. Common dimensions are 512×768 or vice versa. This depends on the models you’re using.

• Model Training and Image Size:

  • Versions likeStable Diffusion 1.5were trained on512×512images. They work well with 768 pixel images as well.
  • However, newer models likeSDXL 1.0,Stable Cascade, andStable Diffusion 3.0are trained on1024×1024images.
  • When generating images using these newer models, sticking to the aspect ratio they were trained on (i.e.,1:1 or 4:3) tends to yield the best results.
  • For example, if you choose a width of512and a height of1024, you’ll maintain the same aspect ratio as the training data.

• Automatic1111 & WebUI Forge:

  • TheHeight/Widthparameters inAutomatic1111allow you to specify the desired image size.
  • To achieve op
    • Width: 512 pixels
    • Height: 1024 pixels

• Batch Size:

  • Batch sizedetermines the number of images generated in a single batch.
  • When you set the batch size, you’re essentially specifying how many images will be processed together in parallel.
  • Larger batch sizes require moreVRAM(graphics card memory) because each image in the batch needs to be processed simultaneously.
  • If your batch size is too high, your graphics card might run out of memory, resulting in an “out-of-memory” error during image generation.
  • Generally,8GB of VRAMis sufficient for Stable Diffusion unless you’re generating very large batches of images.
  • Increasing the batch size can improve efficiency, but it also increases VRAM usage and processing time.
  • Remember that VRAM is separate from your main system RAM and is essential for handling complex AI models.
  • Setting a higher batch size means more parallel image processing, but it also requires more VRAM.

• Batch Count:

  • Batch countspecifies the number of batches of images you want to generate.
  • In the Stable Diffusion WebUI, the batch size can go up to8 images, while the batch count can go much higher (up to100).
  • The total number of images generated is determined by multiplying the batch size by the batch count.
  • For example, if you set the batch count to 10 and the batch size to 1, Stable Diffusion will generate 10 images from the same text prompt.
  • Adjusting the batch count allows you to control the overall number of image generations.

• Purpose:

  • TheCFG Scalecontrols theguidanceprovided to Stable Diffusion during image generation.
  • It plays a crucial role in bothtext-to-image (txt2img)andimage-to-image (img2img)processes.

• Functionality:

  • When you adjust theCFG Scale, you influence how closely the generated image aligns with yourtext prompt.
  • Higher CFG values result in images that closely match the prompt, while lower values allow for more creative deviations.
  • Striking the right balance is essential: too high, and the image may overcomplicate; too low, and it might stray from your intent.

• Experimentation:

  • The ideal CFG Scale varies based on factors like themodeland samplers you’re using and your desired outcome.
  • Some models prefer higher CFG for consistency, while others yield abstract results with lower CFG.
  • Experiment with different values to find what works best for your creative vision.

• Remember, the CFG Scale acts as a balancing tool, shaping the interplay between prompt guidance and artistic exploration!

CFG Scale Balancing

It’s a balancing act to find the right CFG Scale value that provides the best combination of fidelity to the prompt and image quality.Users often experiment with different values to find the sweet spot for their particular project.

• Color Saturation and Contrast:

  • As you adjust the CFG value, you’ll notice changes in color saturation and contrast within the generated images.
  • Higher CFG values often result in more vivid colors and pronounced contrasts.

• Detail and Artifacts:

  • However, there’s a delicate balance to strike.
  • Pushing the CFG value too high can lead to unintended consequences:
    • Loss of Detail: Images may lose fine details, becoming less intricate.
    • Blurrier Output: High CFG values might introduce blurriness.
    • Artifacts: Unwanted artifacts may appear, affecting image quality.

• Scenario Example:

  • Consider using theDPM++ Karrasmodel with20 Sampling Steps(the default Stable Diffusion Web UI settings).
  • Experiment with CFG values to find the sweet spot that aligns with your creative vision.

• Increasing Sampling Steps:

  Adding more sampling steps typically adds more detail to the output image, but be mindful of processing times, which could increase as well.

• Switching Sampling Methods:

  Some sampling methods perform better at specific CFG and sampling steps. For instance, UniPC tends to deliver good results at a CFG as low as 3, while DPM++ SDE Karras excels at providing detailed images at CFG values greater than 7.

Hires steps

When you enableHiRes stepsinAutomatic1111, it reveals a set of sliders that allow you to fine-tune the resolution of your upscaled images. These steps occur after the initial sampling and contribute significantly to the overall image quality. By adjusting these sliders, you can precisely control how much the image is upscaled, ensuring optimal results for your specific requirements.

1. What Are HiRes Steps?
   • HiRes stepsrefine image quality after the initial sampling process.
   • They occurafterthe sampling steps and are responsible for upscaling the image.
   • The total number of steps in the process includes both thesampling stepsand theHiRes steps.
   • You can set theHiRes stepsin the range of0 to 150.
   • If you keep the HiRes steps at0, it means that there are no additional upscaling steps beyond the initial sampling.
     
2. Why Are HiRes Steps Important?
   • HiRes steps allow for further enhancement of image quality.
   • They help refine the denoised image obtained from the sampling process.
   • By adjusting the number of HiRes steps, you can control the trade-off between image quality and computational resources.
   • Note that the impact of HiRes steps on image generation diminishes beyond a certain point.

Denoising Strength

• A value of0has no effect on the image, while a value of1tends to significantly alter the image.
• The denoising strength determines how strongly the upscaler affects image generation.
• By default,Automatic1111sets the denoising strength to0.7.

Upscale by

• The “Upscale By” slider controls the factor by which the image is upscaled during generation.
• By default,Automatic1111sets the width and height at512 x 512pixels.
• When you openHiRes.fix, it’s initially set to‘Upscale by 2’, resulting in a1024 x 1024image.
• You can upscale it by a maximum factor of4x.
• Keep in mind that the more you upscale, the longer it will take to generate your image.

Resize width/Height to

• This setting allows you to specify the target resolution for your image.
• You can either:
  • Set the width and height individually (not recommended).
  • Use the‘Upscale By’slider to achieve the desired resolution.
• Experiment with different options to find the ideal balance between quality and processing time.
• Remember that resizing during generation ensures no artifacts or irregularities are introduced in the image.

HiRes Checkpoint/Sampling Method

InAutomatic1111 version 1.8, two new tabs have been introduced in theHires Fixsection:Hires CheckpointandHires Sampling Method. These tabs provide additional flexibility when upscaling and fixing images.

1. Hires Checkpoint: This feature allows you to select a different checkpoint for the upscaling process. It’s particularly useful if you want to introduce a distinct style to your image during upscaling. You can choose a checkpoint that aligns better with your desired outcome.
2. Hires Sampling Method: The sampling method complements different models. Experimenting with various combinations of checkpoints and sampling methods will help you find the optimal approach for upscaling your images. Additionally, you have the option to prompt the model for specific checkpoints and samplers, providing fine-grained control over the upscaling process.

In summary,HiRes.fixprovides a way to upscale your images during the generation process, improving quality and avoiding post-generation artifacts. Play around with these settings to create stellar images inStable DiffusionusingAutomatic1111!🌟🖼️.

Finding the right balance between denoising and upscaling is essential for achieving optimal results in the AUTOMATIC1111 Stable Diffusion model. Experiment with different settings to achieve the desired output! 🌟

Employing the Hires.fix function effectively can help avoid issues such as twinning and loss of composition in your upscaled images. “Twinning,” in this context, refers to the unwanted duplication or multiplication of features in your creations. For instance, this might result in characters with two faces or two heads, which can be visually disruptive unless intentionally desired.

Restore Faces Button – How does restore faces work in Stable Diffusion?

Restore Faces in Automatic1111

TheRestore Facesfeature inAutomatic1111is designed to enhance and restore facial features in images and videos. Specifically, it focuses on improving the appearance of faces by applying advanced restoration techniques. When enabled, it can significantly enhance the quality of facial details, reduce imperfections, and create more visually appealing results.

Face Restoration Models

1. CodeFormer:
   • This model utilizes a deep learning architecture to restore faces. It is trained on a large dataset of facial images and can handle various lighting conditions, poses, and expressions.
   • The goal of CodeFormer is to produce realistic and high-quality results by learning from diverse face images.
2. GFPGAN:
   • GFPGAN is another face restoration model that uses generative adversarial networks (GANs).
   • It enhances facial features by learning from a wide range of face images, resulting in improved visual quality.

How to Turn On Restore Faces

To enable theRestore Facesfeature inAutomatic1111, follow these steps:

1. OpenAutomatic1111/WebUI Forge.
2. Navigate to theSettingssection.
3. Look for thePostprocessingsettings on the left sidebar.
4. UnderFace Restoration, you’ll find theRestore Facesoption.
5. Click on theRestore Facesbutton to enable it.
6. Apply Settings > Reload UI

CodeFormer Weight Slider

• Theweight sliderallows you to control the intensity of the face restoration effect. By adjusting the slider:
  • Moving it towards the right increases the restoration effect.
  • Moving it towards the left reduces the effect, preserving more of the original facial features.

Move Face Restoration Model from VRAM After Processing

1. After the face restoration process (e.g., applying enhancements to an image or video frame), some software applications choose to move the model from VRAM to RAM.
2. Here’s how it works:
   • During Processing:
     • The model resides in VRAM for fast access by the GPU during active processing.
   • After Processing:
     • Once the processing is complete, the model is moved from VRAM to RAM.
     • By moving the model to RAM:
       • It frees up VRAM for other tasks.
       • It ensures that the model is available for future frames or subsequent processing steps.
       • Both the CPU and GPU can access the model, allowing for more flexibility.

Might be better off sometimes.

In some instances, leaving the ‘Restore Face’ function off could yield better results. On the flip side, if you notice an image where a face appears incomplete or unrefined, that might be a perfect scenario to engage the ‘Restore Face’ feature. In the end, it’s all about experimenting and finding the balance that works best for your creative process. Or you can use After Detailer below for better face detailing.

Understanding the Seed

Seeds are numbers that define the starting point of your image generation in Automatic1111, a GUI for stable diffusion. They control the content of the image, so changing the seed will result in a different image, even if you keep all the other settings the same. Seeds can be set to any integer value, or -1 for a random seed.

You can think of seeds as coordinates in a very large and complex map of possible images. Each seed corresponds to a unique location on this map, and each location has a different image. When you use Automatic1111, you are exploring this map by moving from one location to another, using your text prompt and other settings as guidance.

The seed value is important because it determines what kind of images you will see when you use Automatic1111. For example, if you use the same prompt but different seeds, you will get different variations of the same concept. If you use the same seed but different prompts, you will get different interpretations of the same image. If you use the same seed and the same prompt, you will get the same image every time.

You can find the seed used to generate an image in Automatic1111 by looking at the output window, the filename, or the PNG info tab. You can also change the seed value manually by typing it in the seed box, or clicking the random button to generate a new random seed. By changing the seed value, you can explore different images that match your prompt and settings.

Additional Seed Tools: Dice, Recycle and ‘Extra’

How to use the Seed in Automatic1111/Forge

The Seed is a number that defines the starting point of your image generation in Automatic1111, a GUI for stable diffusion. It controls the content of the image, so changing the seed will result in a different image, even if you keep all the other settings the same. Seeds can be set to any integer value, or -1 for a random seed.

Seed icons

Next to the Seed box, there are a few icons that you should know about:

• The dice icon: This sets your Seed to -1, which means Automatic1111 will choose a random Seed for each image.
• The recycle icon: This copies the Seed from your previous image.

Extra Seed menu

If you want to have more control over the Seed, you can click on the ‘Extra’ option. This will show you the Extra Seed menu with some advanced settings:

• Variation seed: This is another Seed that you can use to modify your image.
• Variation strength: This is a slider that lets you adjust how much of the Variation seed you want to apply to your image. A value of 0 means you only use the original Seed, while a value of 1 means you only use the Variation seed.
• Resize seed from width/height: This is a checkbox that you should enable when you change the size of your image. Otherwise, changing the width or height will also change the content of your image, even if you keep the same Seed. By checking this option, you preserve the content of your image when resizing.

Finding and changing the Seed

You can find the Seed used to generate an image in Automatic1111 by looking at the output window, the filename, or the PNG info tab . You can also change the Seed value manually by typing it in the Seed box, or clicking the random button to generate a new random Seed. By changing the Seed value, you can explore different images that match your prompt and settings.