collapse

Author Topic: Upscaling textures with Artificial Intelligence - ESRGAN, etc  (Read 1681 times)

Offline LPG23

  • Fledgling
  • *
  • Posts: 2
Upscaling textures with Artificial Intelligence - ESRGAN, etc
« on: January 27, 2019, 09:45:38 PM »
Hi everyone, first post here, long time Vampire: Bloodlines fan.

Just recently I discovered these new techniques that upscale low resolution textures to HD using artificial intelligence and the results are INCREDIBLE. I actually tested on Morrowind, Doom and Hexen and it actually looks like a full remaster.

It's some kind of algorithm or magic they call "ESRGAN" that automatically upscales the textures. Apparently, all you need to do is extract the game's textures and apply the algorithm using Python, Nvidia CUDA Cores and some other stuff.

In case you don't know what I'm talking about, check out some of these links (click the screenshots for a better view):

https://www.reddit.com/r/gameupscale

https://www.resetera.com/threads/ai-neural-networks-being-used-to-generate-hq-textures-for-older-games-you-can-do-it-yourself.88272/

https://www.dsogaming.com/news/morrowind-enhanced-textures-is-a-must-have-mod-that-upscales-texture-by-4x-with-esrgan-technique/

https://www.dsogaming.com/news/return-to-castle-wolfenstein-gets-hd-textures-using-esrgan-available-for-download/

The best part is that this method preserves the "original" look of the game.

Unfortunately I don't really have the programming knowledge to make this work (I'm trying though), but if someone made some texture packs with this it would be amazing.



Offline Barabbah

  • Majestic madness maker
  • Antediluvian
  • *****
  • Posts: 1505
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #1 on: January 27, 2019, 11:03:42 PM »
Hello and welcome!
This sounds interesting but I have just one small suspicion: if the textures generated are upscaled this means the files are bigger. If this is the case can this unpolished engine handle this overhaul?
But if you want to try you have to pick up the unofficial SDK (included also in the UP) then extract all the files of the vpk archives, convert all the tth/ttz files to tga, do your algorithm and finally convert those back to tth/ttz. Some texture files are already extracted in the UP with their own corrections.
"Some quotations," said Zellaby, "are greatly improved by lack of context."
― John Wyndham, The Midwich Cuckoos
"fuck the hole in the face"
― me

Offline Barabbah

  • Majestic madness maker
  • Antediluvian
  • *****
  • Posts: 1505
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #2 on: January 27, 2019, 11:07:20 PM »


O my fucking god that's amazing

"Some quotations," said Zellaby, "are greatly improved by lack of context."
― John Wyndham, The Midwich Cuckoos
"fuck the hole in the face"
― me

Offline IanW

  • Methuselah
  • ****
  • Posts: 462
  • Onyx Path Community Manager
    • Onyx Path Publishing
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #3 on: January 28, 2019, 03:45:32 AM »
I've actually tried upscaling on some of the UI elements, but I got a lot of garbage in return. While they seem to work well with larger textures, they don't seem to work very well with the smaller shapes of UI, or perhaps it was the transparencies that had it choking.
Ian A. A. Watson
Onyx Path Community Manager
VTM portal - Tabletop primer

Offline Ghanima_Atreides

  • Libera Me de Ignem
  • Methuselah
  • ****
  • Posts: 312
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #4 on: January 28, 2019, 08:18:44 AM »

This sounds interesting but I have just one small suspicion: if the textures generated are upscaled this means the files are bigger. If this is the case can this unpolished engine handle this overhaul?


There are high-res texture packs out there already, covering most ground/furniture/architecture but personally I don't like them, they are often too shiny/clean but they do work and when I tried them briefly I didn't notice any problems.

This would be WONDERFUL. I would love to try my hand at it if I can figure it out...
Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is
Lost Carcosa.

Offline TheIncredibleKraken

  • Fledgling
  • *
  • Posts: 42
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #5 on: January 28, 2019, 12:09:02 PM »
I was just about to make a post about this topic.

I used https://topazlabs.com/ai-gigapixel/ to achieve these results. Tweaking and mixing with other upscalers will probably be the way to achieve best results.

Original.


Upscaled to 4x on moderate noise reduction and downscaled to 2x.

Offline LPG23

  • Fledgling
  • *
  • Posts: 2
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #6 on: January 28, 2019, 03:40:14 PM »
I was just about to make a post about this topic.

I used https://topazlabs.com/ai-gigapixel/ to achieve these results. Tweaking and mixing with other upscalers will probably be the way to achieve best results.

Original.


Upscaled to 4x on moderate noise reduction and downscaled to 2x.


Wow, this is looking really good!! You can see really see the image has higher res, especially in the eyes and the hair.

From what I read the Topaz Labs AI Gigapixel shows great results, but it's a paid for software.

Also, I actually played Hexen with the "remastered" textures and they used the AI Gigapixel plus Nvidia GameWorks's Super Resolution. The results were really good (https://www.doomworld.com/forum/topic/103770-hexen-neural-texture-pack-23012018/).


Offline Barabbah

  • Majestic madness maker
  • Antediluvian
  • *****
  • Posts: 1505
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #7 on: January 28, 2019, 04:38:09 PM »
I know very probably it's impossible but I'll ask it anyway.
Out of curiosity, it's possible making a script to prompt the game doing the filter at the loading of each texture? Even not only this game, like a SweetFX thing.
"Some quotations," said Zellaby, "are greatly improved by lack of context."
― John Wyndham, The Midwich Cuckoos
"fuck the hole in the face"
― me

Offline TheIncredibleKraken

  • Fledgling
  • *
  • Posts: 42
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #8 on: January 28, 2019, 05:25:24 PM »
I know very probably it's impossible but I'll ask it anyway.
Out of curiosity, it's possible making a script to prompt the game doing the filter at the loading of each texture? Even not only this game, like a SweetFX thing.

The short answer is no.

The long answer is yes, but it wouldn't be worth it. Even if you overcome the programming hurdle that is making the upscaler run on the loading of each texture, it wouldn't work well. a simple 512x512 texture takes several seconds to upscale depending on your gpu. And even when the texture is done upscaling where would it even go? Would you save it somewhere on the hdd/sdd (Making the entire process pointless) or would you store it in ram? If you store it in ram do you discard it after ever level transition(Making you reupscale it later) or keep it on the ram taking more and more space with every texture you upscale?

Offline Barabbah

  • Majestic madness maker
  • Antediluvian
  • *****
  • Posts: 1505
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #9 on: January 28, 2019, 09:52:45 PM »
Point taken. I was hoping for something less difficult to achieve....
"Some quotations," said Zellaby, "are greatly improved by lack of context."
― John Wyndham, The Midwich Cuckoos
"fuck the hole in the face"
― me

Offline Psycho-A

  • Russian Board Moderator
  • Ancillus
  • *
  • Posts: 244
  • Bloodlines SDK developer
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #10 on: January 31, 2019, 11:34:06 AM »
It's pretty impressive!

I tried A.I. Gigapixel too, and results really amazed me. The good side is the program have full-functioning 30 days free trial which enough to recreate all models textures at least :)

Offline Barabbah

  • Majestic madness maker
  • Antediluvian
  • *****
  • Posts: 1505
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #11 on: January 31, 2019, 04:10:08 PM »
And when the 30 days are over  you can continue in another computer :rofl:
"Some quotations," said Zellaby, "are greatly improved by lack of context."
― John Wyndham, The Midwich Cuckoos
"fuck the hole in the face"
― me

Offline Ghanima_Atreides

  • Libera Me de Ignem
  • Methuselah
  • ****
  • Posts: 312
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #12 on: January 31, 2019, 07:14:27 PM »
Oh hell yeah, that looks amazing! Once I am done with my exams I am totally going to give it a whirl.
Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is
Lost Carcosa.

Offline AlienVampire

  • Fledgling
  • *
  • Posts: 9
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #13 on: May 19, 2019, 07:18:52 AM »
Oh hell yeah, that looks amazing! Once I am done with my exams I am totally going to give it a whirl.
Is there any updates?

Offline Ghanima_Atreides

  • Libera Me de Ignem
  • Methuselah
  • ****
  • Posts: 312
Re: Upscaling textures with Artificial Intelligence - ESRGAN, etc
« Reply #14 on: May 19, 2019, 07:56:07 AM »
I'm afraid not. This is actually trickier than it looks and my finals are around the corner. But I still want to give it a try once I'm done with them...unless someone beats me to it. :P
Strange is the night where black stars rise,
And strange moons circle through the skies,
But stranger still is
Lost Carcosa.