Scaler

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

Grabulosaure
Atariator
Atariator
Posts: 26
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Scaler

Postby Grabulosaure » Tue Oct 23, 2018 12:53 am

You like scalers, right ?

I have made one for my cores.

It is not finished, it only does nearest and bilinear, and it is not well tested.
Anyway, here is a preliminary version (ascal.vhd), with the Megadrive/Genesis as an example.

http://temlib.org/pub/mister/ascal/

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue Oct 23, 2018 4:48 am

Looks very interesting! I will try it.
Thanks!

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue Oct 23, 2018 6:22 am

I've tested your precompiled lite version - impressive!
two notes:
- HQ2x produces messed graphics - may be input resolution is beyond expected max..
- I see tearing - tripple frame buffer should solve this problem.

May be you have de-interlacer as well? ;) Some cores like Amiga, TGFX, Genesis require deinterlacer. Especially Amiga.

olin
Atari maniac
Atari maniac
Posts: 81
Joined: Tue Nov 21, 2017 8:57 pm

Re: Scaler

Postby olin » Tue Oct 23, 2018 8:20 am

Would that mean a time unlimited scaler (this new one) could possibly be compiled on a free version of Quartus? Any other benefits like shorter build time of non-lite cores?

vanfanel
Atari User
Atari User
Posts: 42
Joined: Tue Oct 09, 2018 10:19 pm

Re: Scaler

Postby vanfanel » Tue Oct 23, 2018 8:47 am

Sorgelig wrote:- I see tearing - tripple frame buffer should solve this problem.


Triple buffer will add 1 frame of lag. V-synced double buffer is a better solution (unless there's something I'm missing here).

ijor
Hardware Guru
Hardware Guru
Posts: 3594
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Scaler

Postby ijor » Tue Oct 23, 2018 9:12 am

vanfanel wrote:Triple buffer will add 1 frame of lag. V-synced double buffer is a better solution (unless there's something I'm missing here).


I agree. If you are "vsynced" you shouldn't need triple frame buffer. You actually don't need double buffer either if you can "sync" the frame buffer correctly.
Fx Cast: Atari St cycle accurate fpga core

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue Oct 23, 2018 9:40 am

In general case you cannot vsync HDMI to retro video as it usually doesn't follow HDMI requirements.
vsync_adjust is an option for compatible monitors to lock vsync for smooth scrolls but it cannot be a requirement.

vanfanel
Atari User
Atari User
Posts: 42
Joined: Tue Oct 09, 2018 10:19 pm

Re: Scaler

Postby vanfanel » Tue Oct 23, 2018 11:16 am

Sorgelig wrote:In general case you cannot vsync HDMI to retro video as it usually doesn't follow HDMI requirements.
vsync_adjust is an option for compatible monitors to lock vsync for smooth scrolls but it cannot be a requirement.


Of course it can not be a requeriment, but it's one of the coolest things FPGA implementations can do: lagless video. So this option is "the magic" :D

ijor wrote:I agree. If you are "vsynced" you shouldn't need triple frame buffer. You actually don't need double buffer either if you can "sync" the frame buffer correctly.


Exactly!

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue Oct 23, 2018 8:20 pm

Grabulosaure wrote:it only does nearest and bilinear, and it is not well tested.

is it hard to add polyphase filtering? That would make it as replacement of VIP scaler. NN filter is good. Bilinear probably hard to use as it gives too blurry image. Need an option for those who don't like sharp NN look. Also computer cores use text a lot, so NN woudn't be a best choice there.
It would be really wonderful to replace the whole VIP and save a lot of resources with reduced compilation time. ao486 will especially benefit from it.

ijor
Hardware Guru
Hardware Guru
Posts: 3594
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Scaler

Postby ijor » Wed Oct 24, 2018 12:34 am

Sorgelig wrote:is it hard to add polyphase filtering? That would make it as replacement of VIP scaler... It would be really wonderful to replace the whole VIP


A high quality, free, open source, polyphase scaler would be fabulous. Doesn't sound easy. As you said, it requires math skills, probably more than anything else.

May be we should try a joint collaborative effort with some of the mathematician that have been posting in the scaler threads. I googled the issue some time ago. Of course that I couldn't find any open source implementation. But also I found some posts that seem to suggest that the VIP scaler, or at least parts of it, it's written in some math oriented high level language (not on any standard HDL).
Fx Cast: Atari St cycle accurate fpga core

Grabulosaure
Atariator
Atariator
Posts: 26
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Wed Oct 24, 2018 2:21 am

I will first try BiCubic, I think I know how to do it. AFAIU polyphase is not very different.

For interlaced video, is the "field" input sufficient? Decoding composite PAL or NTSC synchro, with equalization pulses, is not fun.

I have updated the Megadrive/Genesis .RBF with support for triple buffering.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Wed Oct 24, 2018 7:12 am

Grabulosaure wrote:I will first try BiCubic, I think I know how to do it. AFAIU polyphase is not very different.

according to Altera docs polyphase uses previous data(pixels?) to calculate current pixel(s) while bicubic use only current data.
Current Genesis version includes coefficients in BRAM and receives it from MiSTer (if supplied), 16 phase with 4 taps. May be you can re-use it in your bicubic algo.

Grabulosaure wrote:For interlaced video, is the "field" input sufficient? Decoding composite PAL or NTSC synchro, with equalization pulses, is not fun.

Yes. Field is enough for determining the second field of interlace video. Progressive video always set it to 0. VIP scaler also use this signal to detect the interlace. Synchro signals aren't used for interlace detection.

Grabulosaure wrote:I have updated the Megadrive/Genesis .RBF with support for triple buffering.

Can you fork my Genesis repository on github and apply your changes with scaler there? It would be easier to track the changes.
If it's not convenient for you then fine. Can use your site.

ijor
Hardware Guru
Hardware Guru
Posts: 3594
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Scaler

Postby ijor » Wed Oct 24, 2018 9:46 am

Grabulosaure wrote:I will first try BiCubic, I think I know how to do it. AFAIU polyphase is not very different.


That would be fabulous!

This is an Altera/Intel article about Bicubic vs Polyphase:
https://www.intel.com/content/www/us/en ... 51468.html
Fx Cast: Atari St cycle accurate fpga core

ghogan42
Atariator
Atariator
Posts: 29
Joined: Wed Oct 17, 2018 7:27 pm

Re: Scaler

Postby ghogan42 » Fri Oct 26, 2018 3:38 am

Grabulosaure wrote:I will first try BiCubic, I think I know how to do it. AFAIU polyphase is not very different.

For interlaced video, is the "field" input sufficient? Decoding composite PAL or NTSC synchro, with equalization pulses, is not fun.

I have updated the Megadrive/Genesis .RBF with support for triple buffering.


Great Work!

I don't have any experience with fpgas, so I don't know how easy/hard anything is to implement. But if you have a working bilinear filter then you might be interested in what I do in my retropie pixel shaders to get improved scaling just using the built-in bilinear filtering hardware.

Here's a screenshot illustrating what I mean (sorry if it doesn't appear in-line):
pixel_scaling.jpg


The picture shows (in order):
  • Regular Bilinear
  • Inigo Quilez's "Improved Texture Interpolation" from here: http://www.iquilezles.org/www/articles/texture/texture.htm
  • The modified sharper version of the above. This is the one I actually use in my pixel shaders
  • An even sharper version that can be used. But this is too close to nearest neighbor and shows shimmering in motion.

I put it in a shadertoy here if you want code: https://www.shadertoy.com/view/Xl2czV

I'll post the algorithm here too. I'm not trying to force you into implementing it. I just figure it's faster to post now then to ask you if you were interested and then waiting for your reply.

Here's how it works. I'm assuming you run the interpolation twice (once in x and once in y):

Code: Select all

Suppose you have a game resolution of 320x240.

Suppose the pixel in the display maps to an x coordinate of 112.13 (just an example).  Also, in pixel shaders we say 112.5 is the center of pixel 112, so x=112.13 would be a distance of 0.37 from pixel center 112 and a distance of 0.63 from pixel center 111.

Instead of linearly interpolating between pixels 111 and 112 with (0.37)(Pixel 111) + (0.63)(Pixel 112) we do this instead:

Adjust the coordinates as follows:
   X = 112.13
   C = floor(X)+0.5  (Center of the current Source Pixel)
   Dist = X - C   (Distance from the Pixel Center)
   Dist = 4.0*Dist^3  (Adjusts the distance from center.  See below for explanation)
   X = C + Dist   (Put the x coordinate back together)

The result from the above would map 112.13 to 112.297

Then you linearly interpolate with standard linear interpolation: (0.203)(Pixel 111) + (0.797)(Pixel 112)


Note that what this is doing is mapping "Dist" from [-0.5,0.5] back onto [-0.5,0.5]. But numbers close to zero get mapped even closer to zero.
And -0.5 maps to -0.5 and 0.5 maps to 0.5 because interpolation would break without that.

You can look here: https://www.desmos.com/calculator/alxmsdemu3 to see how I modify the "Distance from Pixel Center" before linear interpolation. The flatter it is through (0,0) the sharper the scaling

Let me know what you think. Feel free to ignore this and implement whatever you want though. :D
You do not have the required permissions to view the files attached to this post.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Fri Nov 02, 2018 6:03 am

Grabulosaure wrote:I will first try BiCubic, I think I know how to do it. AFAIU polyphase is not very different.

is there any progress?

cacophony
Atari maniac
Atari maniac
Posts: 85
Joined: Sun Jul 22, 2018 11:14 pm

Re: Scaler

Postby cacophony » Fri Nov 02, 2018 5:09 pm

Part of the motivation for using a custom scaler:
https://twitter.com/RetroRGB/status/1058093654554894336

According to RetroRGB:
"I did some MiSTer lag tests. In it's current state, the MiSTer's HDMI output ads about 2 frames of lag. Future community-built scalers can significantly lower this though."

Grabulosaure
Atariator
Atariator
Posts: 26
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Sat Nov 03, 2018 12:26 am

Sorgelig wrote:is there any progress?


I was away from Internet for a few days... But I had my computer!

Here is the current state of the scaler :

http://temlib.org/pub/mister/ascal/

Big changes. Bugs.

BlockABoots
Atari freak
Atari freak
Posts: 51
Joined: Sat Oct 20, 2018 3:20 pm

Re: Scaler

Postby BlockABoots » Sat Nov 03, 2018 12:37 am

So this scaler can be implemented into and core, or is it just Genesis?

coughlanio
Retro freak
Retro freak
Posts: 12
Joined: Fri Oct 06, 2017 6:15 pm

Re: Scaler

Postby coughlanio » Sat Nov 03, 2018 2:22 am

I've noticed some severe image retention with the Genesis core using the new scaler on my monitor. It's an LG 27UK850W. I tried to see if I could reproduce the burn-in with the standard Genesis core, but it seems to be exclusive to the new scaler. Speaking with some people, it could be due to weird video timings?

Grabulosaure
Atariator
Atariator
Posts: 26
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Sat Nov 03, 2018 3:39 am

I have just fixed a bug and updated the files.

coughlanio wrote:I've noticed some severe image retention with the Genesis core using the new scaler on my monitor. It's an LG 27UK850W. I tried to see if I could reproduce the burn-in with the standard Genesis core, but it seems to be exclusive to the new scaler. Speaking with some people, it could be due to weird video timings?


What do you mean by image retention? Do the "triple buffer" option change anything ? The scaler should use the same video modes as Sorgelig's Genesis core.

BlockABoots wrote:So this scaler can be implemented into and core, or is it just Genesis?


It can be used with any core, replacing the Intel/Altera scaler.

cacophony wrote:According to RetroRGB:
"I did some MiSTer lag tests. In it's current state, the MiSTer's HDMI output ads about 2 frames of lag. Future community-built scalers can significantly lower this though."


This scaler doesn't try to minimise lag yet. It shouldn't be too difficult, assuming that the output vertical frequency is locked to the input frequency. Currently, with the "triple buffer" mechanism, there is a 1 to 2 images delay.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Sat Nov 03, 2018 6:43 am

Grabulosaure wrote:I have just fixed a bug and updated the files.

huge progress!
There are image corruptions on edges - may be it's possible to extend internal buffer to include the blanking area and then cut it on output, so these artifacts will be outside of active area. Is possible to add Width/Height of active image within a current resolution with adding black borders? It's needed for aspect ratio compensation and integer vertical scaling.

ghogan42
Atariator
Atariator
Posts: 29
Joined: Wed Oct 17, 2018 7:27 pm

Re: Scaler

Postby ghogan42 » Sat Nov 03, 2018 8:54 am

Grabulosaure wrote:I have just fixed a bug and updated the files.

coughlanio wrote:I've noticed some severe image retention with the Genesis core using the new scaler on my monitor. It's an LG 27UK850W. I tried to see if I could reproduce the burn-in with the standard Genesis core, but it seems to be exclusive to the new scaler. Speaking with some people, it could be due to weird video timings?


What do you mean by image retention? Do the "triple buffer" option change anything ? The scaler should use the same video modes as Sorgelig's Genesis core.

BlockABoots wrote:So this scaler can be implemented into and core, or is it just Genesis?


It can be used with any core, replacing the Intel/Altera scaler.

cacophony wrote:According to RetroRGB:
"I did some MiSTer lag tests. In it's current state, the MiSTer's HDMI output ads about 2 frames of lag. Future community-built scalers can significantly lower this though."


This scaler doesn't try to minimise lag yet. It shouldn't be too difficult, assuming that the output vertical frequency is locked to the input frequency. Currently, with the "triple buffer" mechanism, there is a 1 to 2 images delay.


I know you have more interesting things to work on. Good job on all you've done! But I notice that the method I described isn't working. I don't know VHDL but based on how the vip scaler seems to work, a 4bit LUT should implement my function well enough. I'm just going to copy/paste the function here for you. I tried it and it works correctly.

Code: Select all

  -----------------------------------------------------------------------------
  -- Ghogan
  -- Shift Bilinear weights away from the center (towards 0 or 1) to
  -- make scaling sharper
  FUNCTION gho_calc(frac : unsigned(11 DOWNTO 0)) RETURN unsigned IS
  BEGIN
     case frac(11 DOWNTO 8) is  --We don't need so many bits
      when "0000" => Return "00000000";
      when "0001" => Return "00000001";
      when "0010" => Return "00000010";
      when "0011" => Return "00000110";
      when "0100" => Return "00010000";
      when "0101" => Return "00100000";
      when "0110" => Return "00110110";
      when "0111" => Return "01010101";
      when "1000" => Return "10000000";
      when "1001" => Return "10101010";
      when "1010" => Return "11001010";
      when "1011" => Return "11100000";
      when "1100" => Return "11110000";
      when "1101" => Return "11111010" ;
      when "1110" => Return "11111110";
      when "1111" => Return "11111111";
    end case;
  END FUNCTION;
 
  SIGNAL o_h_gho_pix,o_v_gho_pix : type_pix;
  SIGNAL o_h_gho_frac,o_v_gho_frac : unsigned(7 DOWNTO 0);



Also, it's worth noting that the way the vip polyphase filter works similarly with a 4bit lut. I think it's pretty much like this for horizontal scaling. Does this sound right to you? I'm still wrapping my head around it.

-----------------------------------------------------------------
VIP Polyphase Filter with 4 taps and 16 Phases?
-----------------------------------------------------------------
(1) (Currrent Output X)/(Output X Res) * (Input Res X) - 0.5 = Position
(2) Split Position into Integer Part and Fractional Part -- I'll call them Int_Pos and Frac_Pos
(3) Quantize Frac_Pos down to 4 bits if it isn't already
(4) Use these four bits to look up a row in the coefficient table. Call them C0, C1, C2, C3. Use however many bits you want. These need to be signed. Example Table For Bicubic Scaling here:

"0000": 0, 128, 0, 0
"0001": -4, 127, 5, 0
"0010": -7, 124, 12, -1
"0011": -10, 119, 21, -2
"0100": -11, 112, 31, -4
"0101": -11, 103, 41, -5
"0110": -11, 94, 52, -7
"0111": -11, 84, 63, -8
"1000": -10, 74, 74, -10
"1001": -8, 63, 84, -11
"1010": -7, 52, 94, -11
"1011": -5, 41, 103, -11
"1100": -4, 31, 112, -11
"1101": -2, 21, 119, -10
"1110": -2, 13, 124, -7
"1111": 0, 5, 127, -4

(5) Fetch Pixels T0, T1, T2,, T3 from the source buffer. Pixel T1 is at Int_Pos. So T0 is to the left and T2 and T3 are to the right. Duplicate pixels if you run off the edge of the buffer?
(6) Your Output Pixel Color is C0*T0 + C1*T1 + C2*T2 + C3*T3
---------------------------------------------------------------------
Can anybody see any mistakes that I made? Since all of the useful filters can be implemented with a 4 tap filter like this, I might try adding it as an extra option to the ascal scaler. I might not succeed because simple things in VHDL are hard for me since I have never used it before today. But if it works then we have all of the filters that I made over in the other thread: http://www.atari-forum.com/viewtopic.php?f=117&t=34430&start=100#p356779 in addition to the ones implemented directly in ascal.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2906
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Sat Nov 03, 2018 11:06 am

I believe all these filters including bilinear can be implemented by just different coefficients.
Recent MiSTer binary already use coeff.txt file from Genesis folder and push them to the core.
So you don't need to hardcode coefficients in HDL. Just receive them like it's done in my recent Genesis core.

kitrinx
Retro freak
Retro freak
Posts: 14
Joined: Wed Sep 26, 2018 6:03 am

Re: Scaler

Postby kitrinx » Sat Nov 03, 2018 1:43 pm

The menu option for changing the coefficients is actually a pretty nice user experience feature. The coeff.txt method provides the maximum flexibility and the ability to experiment and grow, but having that menu option with those presets makes life a lot easier, especially for less experienced users. Maybe some middle ground could be found?

coughlanio
Retro freak
Retro freak
Posts: 12
Joined: Fri Oct 06, 2017 6:15 pm

Re: Scaler

Postby coughlanio » Sat Nov 03, 2018 1:44 pm

Grabulosaure wrote:
coughlanio wrote:I've noticed some severe image retention with the Genesis core using the new scaler on my monitor. It's an LG 27UK850W. I tried to see if I could reproduce the burn-in with the standard Genesis core, but it seems to be exclusive to the new scaler. Speaking with some people, it could be due to weird video timings?


What do you mean by image retention? Do the "triple buffer" option change anything ? The scaler should use the same video modes as Sorgelig's Genesis core.


Image retention still occurs when using triple buffered mode.

Here's an example of what I can see when I switch back to my desktop:
Image

It seems to resolve itself after a minute or two.


Return to “MiSTer”

Who is online

Users browsing this forum: Indcsion, Slade and 9 guests