Using AI to Draw Realistic Images - Github Project - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Sunday, March 26, 2023

Using AI to Draw Realistic Images - Github Project

ChatGPT has been out for a while, which raises lots of discussion and interesting on using those artificial intelligence technologies to complete some work orginal done by human. Using AI to draw a realistic photo is another interesting topic which has been deeply developped for certain purpose. In this post, I will show a Github project which you can use it to create your own text-2-image Web UI page then use free Google Colab platofrm to install. You also are able to install it into your local pc  to draw the pictures based on your inputs, either a image or some texts, but you had better to have a good performance machine, especially high end graphic card. 


Introduction

Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images given any text input.



Model Details

More details can be found : https://huggingface.co/CompVis/stable-diffusion-v-1-4-original

Github Project: stable-diffusion-webui-colab


Here are steps to follow this Github project to create your own WebUI on Colab to run: 

1 Go to Github Page : https://github.com/camenduru/stable-diffusion-webui-colab

2 Got to Colab section, select one option to run


Click one button to continue to load it in Colab website.

3 Colab site with loaded code.


Click run button to run those commands in Colab environment. 

4 Once completed the running, you can scroll down to find out those two WebUI links:


5 Stable Diffusion WebUI page will be opened:


6 You can enter keywords, sentences or other related words to describe the photo you wants, then click generate to see the output.

You can generate it multiple times to see different results.


Prompt Examples

Example 1: 

In a way you gonna have experience as an artist to know what make art look good.

I want a VICTORIAN GIRL in a style of OIL PAINTING
Eye and Face are important in art so she must have PERFECT FACESEXY FACE and her eye have DETAILED PUPILS
I want she to have LARGE BREASTTONED ABS and THICK THIGH.
She must look FEMININE doing EVOCATIVE POSESMIRK and FULL BODY wearing NIGHT GOWN
The output must be INTRICATEHIGH DETAILSHARP
And in the style of {I'm not give out the artist names to avoid trouble. Apologize.}

Prompt :
VICTORIAN GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),ARTIST,ARTIST,(ARTIST). OIL PAINTING. (((LARGE BREAST)),((TONED ABS)),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP

This is where you put trait that you dont want to see in the finale output. i.e nipple
Negative Prompt :
((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))

TIPS:

  • prompt in () will strengthening the prompt, adding more will strengthening it even further
  • prompt in [] will weakening the prompt , adding more will weakening it even further
  • Adding more artist will create amalgamation that end up look better or worst, experiment it yourself. I use 4 artist to create most of the output.
  • Restore Face is GOD SEND, USE IT.
  • It always depend on what model you use.

The level here is whatever, feel free to experiment. This is what I use.
Steps: 42, Sampler: DDIM, CFG scale: 10

Example 2 : Manga style

Prompt : GIRL,FEMININE,((PERFECT FACE)),((SEXY FACE)),((DETAILED PUPILS)).(ARTIST),(THICK THIGH).EVOCATIVE POSE, SMIRK,LOOK AT VIEWER, ((BLOUSE)).(INTRICATE),(HIGH DETAIL),SHARP,(((loli))) 

Negative Prompt : ((nipple)), ((((ugly)))), (((duplicate))), ((morbid)), ((mutilated)), (((tranny))), (((trans))), (((trannsexual))), (hermaphrodite), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))). (((more than 2 nipples))). [[[adult]]], out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck)))


Local Installation of Stable Diffusion Web UI

GitHub Page: https://github.com/automatic1111/stable-diffusion-webui/

Steps:

  1. Install Python 3.10.6, checking "Add Python to PATH".
  2. Install git.
  3. Download the stable-diffusion-webui repository, for example by running git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git.
  4. Close GUI interface. 
  5. Download models (model.ckpt) file,  and put them under models directory.  
  6. Run webui-user.bat from Windows Explorer as normal, non-administrator, user.

http://127.0.0.1:7860/


Generate a photo:


Total folder size: 9.40GB


Error message:

you might be getting some python related error. You can quickly update pip and reboot your system to try it again. 


Fix: reboot system and try it again.


Another error message might relate to your GPU. 

AssertionError: Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check. 

If your system does not have much RAM , you might want to add som other parameters to avoid installing issues:

 "set COMMANDLINE_ARGS = --lowvram --precision full --no-half --skip-torch-cuda-test"



Models

Popular Models:



Model: Cocoa.ckpt , yohan-diffusion.ckpt

Open In ColabOpen In ColabOpen In Colabyohan_diffusion_webui_colab
andite/yohan-diffusion

!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/yohan-diffusion/resolve/main/yohan-diffusion.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o yohan-diffusion.ckpt
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/sd-vae-ft-mse-original/resolve/main/vae-ft-mse-840000-ema-pruned.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o yohan-diffusion.vae.pt
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/ckpt/yohan-diffusion/resolve/main/Cocoa.ckpt -d /content/stable-diffusion-webui/models/Stable-diffusion -o Cocoa.ckpt


Prompt: a beautiful 18 years old asian girl, smile face, sitting in front of a house, looking at you, Big eyes

Model Stable Diffusion v2.1

Open In ColabOpen In ColabOpen In Colabstable_diffusion_v2_1_webui_colab
stabilityai/stable-diffusion-2-1

It is using Stable Diffusion checkpoint : 768-v-ema.ckpt 

Prompt: a beautiful 18 years old asian girl, smile face,  sitting in front of a house, looking at you, Big eyes

Negative Prompt: lowres, low quality, doll, bad anatomy, poorly drawn, ugly, maiden, virgin, malformed, deformed, masculine, manly, black skin, asian, blurry, out of focus, freckles, skin defects, skin marks, long neck, anime, highlights, shadows,





Some Sites to Use AI

 
The list which collected from Internet will be kept updating from time to time:




LoRA and Checkpoint

What is LoRA?


LoRA models are small Stable Diffusion models that apply tiny changes to standard checkpoint models. They are usually 10 to 100 times smaller than checkpoint models. That makes them very attractive to people having an extensive collection of models.


LoRA models are small modifiers of checkpoint models. You can easily use them in AUTOMATIC1111 by including a phrase in the prompt.

Civitai hosts a large collection of LoRA models. Apply the LORA filter to see only LoRA models. You may find that they all tend to be similar: female portraits, anime, realistic illustration styles, etc.

Hugging Face is another source of LoRA libraries. You will find more varieties of LoRA models. But there are not as many LoRA models there. Their collection is a lot smaller.

Search LoRA models in Hugging Face

LoRA Library in Hugging Face


Usage:

As an example, I’ll use Arcane LoRA and download the .safetensors file. When the file is finished downloading, put it in stable-diffusion-webui/models/Lora.

!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/8339 -d /content/stable-diffusion-webui/models/Lora -o ArcaneLoRA.ckpt

For the SD checkpoint: You can use any of your existing models, but for guaranteed pretty good results download the AnyLoRA model. Like the rest of your models, this goes in the folder stable-diffusion-webui/models/Stable-diffusion.

!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://civitai.com/api/download/models/29792 -d /content/stable-diffusion-webui/models/Stable-diffusion -o AnyLoRA.ckpt


Lets load the model and checkpoint into Colab throught Github project's link:
Open In ColabOpen In ColabOpen In Colabopenjourney_v2_diffusion_webui_colab
prompthero/openjourney-v2





Prompts

提示词(Prompt)
  • Digital painting of a charming girl sitting on the sofa, extremely detailed, ornate, cinematic lighting
  • Oil Painting(水彩画),Digital Painting(数字绘画),Abstract Painting(抽象画)
  • 可以用 "()" 增加词重要性,用 "[]" 降低词的重要性,
  • “vivid”,这个词汇可以让你的图片显得灵动, 更有质感,像电影一样,可以试试这个 “Ornamented, fancy, motion-picture lighting”
  • 通过在提示词使用符号"[]",可以生成融合脸,公式大概可以概括为[A: B: 0-1之间的数],比如下面这个例子:[Donald Trump: Joe Biden: 0.5]
  • 控制图片中的人物视角,可以参考以下词组:looking at the camera,look left,look right,full body

反义提示词(Negative)
  • text, signature, title, heading, watermark, ugly, duplicate, morbid, mutilated, out of frame, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, out of frame, ugly, extra limbs, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, mutated hands, fused fingers, too many fingers, long neck

Videos

 



References


No comments:

Post a Comment