LoRA

https://civitai.com/models/1266100/the-incredible-hulk-2008, LoRA The Incredible Hulk (2008)

On his personal page on the CivitAI website, the user BigHeadTF promotes his recent creation, a small model called The Incredible Hulk (2008). Compared to earlier movies of the Hulk, the 2008 version shows a tormented Bruce Banner who transforms into a green creature with "detailed musculature, dark green skin, and an almost tragic sense of isolation". The model helps generate characters resembling this iconic version of Hulk in new images.

To demonstrate the capabilities of his model, BigHeadTF has selected a few pictures of his own. Hulk is in turn depicted as cajoling a teddy bear or crossdressing as Shrek's Princess Fiona. The images play with the contrast between Hulk's overblown virility and childlike or female connotations. The images demonstrate the model's ability to expand the hero's universe into other registers or fictional worlds. The Incredible Hulk (2008) doesn't just reproduce faithfully existing images of Hulk, it also opens new avenues for creation and combinations for the green hero.

This blend of pop and remix culture that strives on the blurring of boundaries between genres infuses a large number of creations made with generative AI. However what distinguishes BigHeadTF is that he doesn't share only images but the software component that makes his images distinctive. The model he distributes on his page is called a LoRA. The most famous models such as Stable Diffusion or Flux are rather general-purpose. These 'base' or 'foundation' models can be used to generate images in many styles and can handle a huge variety of prompts. But they may show limitations when a user wants a specific output such as a particular genre of manga, a style that emulates black and white film noir or when an improvement is needed for some details (specific hands positions, etc) or to produce legible text. This is where LoRAs come in. A LoRA is a smaller model created with a technique that makes it possible to improve the performance of a base model on a given task.

What is a LoRA?

Initially developed for LLMLow-Rank Adaptation of Large Language Models. Technically the LoRA freezes an existing model and insert a smaller number of weights to adjust the model's behaviour to a particular need. Instead of a full retraining of the model, LoRAs only require the training of the weights that have been inserted. Therefore LoRAs are quite lightweight and able to leverage the capabilities of larger models. Users equipped with a consumer-grade GPU can train their own LoRAs reasonably fast (on a mac M3, a LoRA can be produced in 30 minutes). LoRAs are quite popular within communities of amateurs and developers alike. At the time of writing, the AI platform Hugging Face lists 71,312 LoRAs.

What is the network that sustains this object?

• Free software libraries and apps
  • https://huggingface.co/docs/peft/index
  • https://github.com/bmaltais/kohya_ss/
  • https://civitai.com/articles/3168/lora-inspector
• Tutorials and documentations
  • https://education.civitai.com/lora-training-glossary/
  • https://civitai.com/articles/4/make-your-own-loras-easy-and-free
  • https://civitai.com/articles/8310/lora-training-using-illustrious-report-by-a-beginner Since Illustrious learned on danbooru, she has memorized many of the Art Styles of the artists who have contributed to danbooru. Therefore, it is better to check if it is already learned Art Style you want to resemble, so that you do not have to waste your time. Note that I wasted several hours trying to create the Art Style of my favorite doujinshi and eroge illustrators
• Technical communities and Manga fans
  • https://civitai.com/articles/197/not-maintained-anymore-use-tag-search-instead-arknights-lora-collection-in-alphabetical-order
  • https://civitai.com/articles/166/lora-training-guide-rz-passage-21-15
• Dependency on base models: This LoRa model was designed to generate images that look more "real" and less "artificially AI-generated by FLUX". It achieves this by making the natural and artificial lighting look more real and making the bokeh less strong. It also adds details to every part of the image, including skin, eyes, hair, and foliage. It also aims to reduce somewhat the common "FLUX chin and skin" and other such issues. https://civitai.com/models/970862/amateur-snapshot-photo-style-lora-flux

How does it evolve through time?

• From the Microsoft lab to platforms and informed amateurs, diversification of offer
• Expansion of the image generation pipeline

How does it create value? Or decrease / affect value?

Rewards for the best creative use of a LoRA, https://civitai.com/bounties/8690/5k-crazywhatever

• Adds value to the base model. Combined with the LoRA, its capabilities are expanded
• Different forms of value creation -> cultural
• Huge popularity, there are 66,846 LoRAs available on Hugging Face https://huggingface.co/models?sort=trending&search=lora
• The Mass-Produced LoRAs of Civitai https://civitai.com/articles/10068/the-mass-produced-loras-of-civitai
• 🌟 Do you want your own LoRA created by me? 🌟 https://civitai.com/articles/13480/do-you-want-your-own-lora-created-by-me
• Bounties and LoRAs https://civitai.com/articles/16214/new-bounties-and-loras

What is its place/role in techno cultural strategies?

• Curation, classification, defining styles
• Demystify the idea that you can generate anything
• Exploiting the bias, reversing the problem, work with it
• Conceptual labour
• Sharing
• Visibility in communities
• Needs are defined bottom-up

How does it relate to autonomous infrastructure?

• Regain control over the training, re-appropriation of the model via fine-tuning
• (Partial) Decentralization of model production
• The question of the means of production
• Ambivalence

Images

Two images generated with the prompt "Student apartment in Budapest with mezzanine". Image on the right has been generated with a LoRA trained with images of mezzanines

The LoRA decomposition as explained by a coder

Screengrab of the LoRA page on the civit.ai platform

LoRA: How to Adapt LLMs Efficiently and Without Latency, By Elisa Terumi, https://exploringartificialintelligence.substack.com/p/lora-how-to-adapt-llms-efficiently