waifu2x - deep convolution software for image resize/denoise

piggsy · Joined: 04 May 2015 Posts: 84 Location: Brisbane, Australia

Just wanted to share this neat toy -

http://waifu2x.udp.jp/

it's kind of along the same lines as neural styles, deep dream, prisma, etc, which you may have heard of before. Only rather than repeatedly trying to match images to an artistic style over many iterations, this one tries to draw clean lines and (optionally) de-noise an image.

Probably best to use their own example image to show what it does -

as in its suggested use case it does a fantastic job of resizing compressed images of drawn art with lots of bold colours and strong lines. And it's really designed to hold up just for a goof image where you you have a bit of dvd or blu ray captured and want to make it visible at non-ant sizes. But if you start feeding it lossless images (or better, lossless unsharpened images) and keep on scaling, it can do some remarkable things.

Some examples -

100% - original image (1920x1200)

100% - 8x scaled image (15360x9600)
http://abload.de/img/dejesus100-8xlysgc.png

100% - original image (1151x1600)

100% - 8x scaled image (9208x12800)
http://abload.de/img/gorobo-100-8xvfurs.png

If you manage to find uncompressed sources of an image made of clearly defined lines you're pretty much only limited by memory requirements (much past ~10k x 15k pics do my 4GB 970GTX/16GB RAM box in). Although if you manage to de-jpg well enough to begin with you can push a jpg very far also, at least, probably far enough for most non-billboard purposes.

Here's how it handles a regular image of mine -

feed it the TIFF source (3411x2625)

100% - source

100% - 4x scaled output (13644x10500)

https://abload.de/img/clipboard01yvzgz.jpg

That's the simplest possible scaling too, without going in and doing anything differently noise or sharpening wise, just, asking it to blow it up. That's pretty neat.

Anyway, for most people who aren't prepared to install a linux, there's a web portal version of it at the link with a small-ish file output limit, and if you hit their github, you'll find a windows executable version of it with an installable GUI and right click "send to" menu support (look in waifu2x-converter_x64_1130\ExtendedSendTo) here:

https://github.com/tanakamura/waifu2x-converter-cpp

ed- just to add, this is one of the more lightweight neural network photography things going - even if you don't have a CUDA or OpenCL/GCN supported GPU, you can still make very big images in reasonable timeframes on the CPU only. About 15-40 minutes depending on the source and output on my Haswell/DC I5 at 4.3ghz, vs about 0.5/1 minute on the 970.

kds315* · Expire: 2021-03-09

piggsy · Joined: 04 May 2015 Posts: 84 Location: Brisbane, Australia

One other thing that makes this handy - other than just being able to enlarge a lot of image types - is what happens when you combine this with other neural net programs.

The fastest one I know of right now is Prisma -

http://prisma-ai.com/

which is an ios/android cloud app that, while it doesn't let you use custom style files, has more or less enough of them to have fun in their kit, and it lets you fine tune the effect strength instantly. Most others involve installing a bunch of packages in linux, several minutes of waiting, and often have pretty poor output sizes without a lot of VRAM to throw at them (about 800-900px square was about all I could get out of my 970).

Prisma is kind of crappy output wise, unfortunately - it works good enough for your phone and for a goof, but it's hard to get something like a "serious" image out of it - about 1080px square and 80% quality jpg is all it will spit out. Fortunately you aren't also just limited to your smartphone's camera, you can upload any file sent over USB or on a smart card also. Here's just a random photo sent through it -

Pretty cool effect. But on closer inspection you can see how bad it is at a low level -

here's what waifu2x does to that at 8x scaling and 2 factor noise reduction -

https://abload.de/img/cleanoutukqrv.jpg

pretty neat huh. Full size -

https://abload.de/img/78ab87e7a3df0bc4742b6czssg.jpg

Some others - prisma app out, original file sent to prisma, waifu2x'd output at 8x_N2. I should also mention, I'm just using 8x to show how ridiculous an enlargement factor you can use and how good it will still look given a crappy jpg that was already a "neural net done quick" fast/low quality image, there's no reason to go to a 55mb png at 8000px size if 2x would satisfy your output requirement.

https://abload.de/img/pa010082thituoj.jpg

https://abload.de/img/1dd66cd063887318e467cbzss2.jpg

https://abload.de/img/p1290136raung.jpg

https://abload.de/img/2e5137f7371faccc574d521s2m.jpg

https://abload.de/img/p1280097th7fuh6.jpg

https://abload.de/img/111111111111111111111ncs9e.jpg

https://abload.de/img/p1180227t82u0g.jpg

https://abload.de/img/111111111111111111111f8sv1.jpg

https://abload.de/img/p4140014tht8uix.jpg

https://abload.de/img/111111111111111111111q3s03.jpg

https://abload.de/img/p4100124thpfsm7.jpg

https://abload.de/img/b7b9d7335e4bf67ad0c351bsdu.jpg

https://abload.de/img/p9280101th2lukt.jpg

https://abload.de/img/ff333dea9152b0979c55ciisw1.jpg

bghomofaber · Joined: 03 Mar 2016 Posts: 154

For many deconvolution, sharpening and a myriad other cool filters (including film emulation) I use g'mic filter packs with gime experimental from partha.com (both as appimage for linux and for windows).

For deconvolution and resize I also use Michael's Winter Image Analyzer from meesoft.dk.

I will surely try the suggested program, ty for posting

bghomofaber · Joined: 03 Mar 2016 Posts: 154

Will also post my linux (fedora) compilatione xperience from the github software

https://github.com/nagadomi/waifu2x

my guess is that it will be a good battle ...

piggsy · Joined: 04 May 2015 Posts: 84 Location: Brisbane, Australia