NVDI & character mapping

GFA, ASM, STOS, ...

Moderators: simonsunnyboy, Mug UK, Zorro 2, Moderator Team

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

NVDI & character mapping

Postby ThorstenOtto » Thu Dec 14, 2017 7:39 am

Hi,

i'm currently experimenting with TrueType fonts, and their mapping from Atari ST encoding to unicode. As it turns out, NVDI seems to use a different encoding for Hebrew characters (range 0xc0-0xdc at least) in Vector fonts, as seen in this picture
Image

The left half is the system character set, the right half Arial (Font id 9908).

So i thought, to get the actual encodings, i should use vqt_char_index(). However, no matter what i try, i always get unicode index 0xffff (character not available), ie. in this loop:

Code: Select all

for (i = 0; i < 256; i++)
   unicode = vqt_char_index(vdihandle, i, 0, 2);


What am i doing wrong?

User avatar
christos
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2367
Joined: Tue Apr 13, 2004 8:24 pm
Location: Greece
Contact:

Re: NVDI & character mapping

Postby christos » Thu Dec 14, 2017 7:48 am

From my user experience the mapping is 1-1. That means that NVDI will just map the first 255 characters in the font to the Atari character set so if you open the font in something like fontforge you will see it as what you display in your image. I don't think NVDI supports unicode at all.
Felix qui potuit rerum cognoscere causas.
My Atari blog

STOT Email address: stot(NoSPAM)atari(DOT)org

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Thu Dec 14, 2017 8:30 am

Hm. There must be done some mapping at least, since the font does not have any glyphs in the range 0x80-0x9f for example. Also, the vqt_char_index() function is documented since NVDI 3 i think; would be strange to document, but not implement it.

mikro
Atari God
Atari God
Posts: 1618
Joined: Sat Sep 10, 2005 11:11 am
Location: Kosice, Slovakia
Contact:

Re: NVDI & character mapping

Postby mikro » Thu Dec 14, 2017 9:46 am

I hope I wont say something totally dumb but there used be a tool called Charmap or so and it allowed you to make a totally custom font mappings. I had used it to create local Czech/Slovak TTF map for GEM apps.

EDIT: Yup, this is it: https://milan.kovac.cc/atari/software/?folder=/FONT (look for Charmap)

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Thu Dec 14, 2017 10:19 am

Thanks for the hint, but i have that program already ;) Also most other Font related utilities. Charmap does not seem to work though, it does not find any usable fonts, maybe it only works with SpeedoGDOS. Also, that is not what i need, i don't want to redefine any mapping. What i want is to do text output using unicode indexes somehow (which btw works with NVDI:
Image

For that to work, i need to know which characters are available, or at least to which character index they are mapped when using the default mapping.

User avatar
jfl
Atari Super Hero
Atari Super Hero
Posts: 856
Joined: Tue Jul 18, 2006 10:55 pm
Location: Liège, Belgium
Contact:

Re: NVDI & character mapping

Postby jfl » Thu Dec 14, 2017 11:40 am

ThorstenOtto wrote:Thanks for the hint, but i have that program already ;) Also most other Font related utilities. Charmap does not seem to work though, it does not find any usable fonts, maybe it only works with SpeedoGDOS. Also, that is not what i need, i don't want to redefine any mapping. What i want is to do text output using unicode indexes somehow (which btw works with NVDI:
Image

About the bolded: I'm not sure I understand what you mean here. How does that picture demonstrate that NVDI handles Unicode? Edit: I'm not saying it doesn't, since NVDI most certainly handles Unicode fonts fine, just that I don't see how that picture demonstrates it.

Edit 2: Never mind the above; I had missed the Hebrew characters in the screenshot.
Last edited by jfl on Thu Dec 14, 2017 3:25 pm, edited 1 time in total.
Jean-François
GEMDict – GEMClip

User avatar
Rajah Lone
Captain Atari
Captain Atari
Posts: 392
Joined: Wed Aug 07, 2002 12:27 pm
Location: Lyon / France
Contact:

Re: NVDI & character mapping

Postby Rajah Lone » Thu Dec 14, 2017 3:01 pm

It's old, sorry if my memory fails: Unicode is supported in NVDI 5 (not in 3).

I have study this subject a little, in order to support i18n in my atari email client Troll: http://ptonthat.fr/tiniconv-ldg/

If you can read the GFA source, you'll find some hints.

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Thu Dec 14, 2017 3:30 pm

jfl wrote:I'm not sure I understand what you mean here. How does that picture demonstrate that NVDI handles Unicode?


Sorry, that was not clear of course. It is the same font used as in first example (Arial). But instead of using v_gtext() for the output, i was using a mapping table (based on ftp://www.unicode.org/Public/MAPPINGS/V ... TARIST.TXT), invoked vst_map_mode(vdihandle, 2), and then output the mapped values using v_gtext16(). The result is, that the hebrew characters are shown in the range 0xc0-0xdf, instead of some multi-lingual characters that you see in the first example. v_gtext16() is actually the same VDI function as v_gtext(), just a different binding allowing 16bit values to be passed.

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Thu Dec 14, 2017 3:37 pm

Rajah Lone wrote: Unicode is supported in NVDI 5 (not in 3).


Thats possible. I was just looking at TOS.HYP, where some of the relevant functions are reported by available since NVDI 3. But i'm actually using NVDI 5.03, so that is not problem. And in general, output works, just that vqt_char_index() does not.

Rajah Lone wrote:Troll: http://ptonthat.fr/tiniconv-ldg/

If you can read the GFA source, you'll find some hints.


Yes, thank you, i'll have a look at it.

User avatar
jfl
Atari Super Hero
Atari Super Hero
Posts: 856
Joined: Tue Jul 18, 2006 10:55 pm
Location: Liège, Belgium
Contact:

Re: NVDI & character mapping

Postby jfl » Thu Dec 14, 2017 6:56 pm

ThorstenOtto wrote:
Rajah Lone wrote: Unicode is supported in NVDI 5 (not in 3).


Thats possible. I was just looking at TOS.HYP, where some of the relevant functions are reported by available since NVDI 3. But i'm actually using NVDI 5.03, so that is not problem. And in general, output works, just that vqt_char_index() does not.

Rajah Lone wrote:Troll: http://ptonthat.fr/tiniconv-ldg/

If you can read the GFA source, you'll find some hints.


Yes, thank you, i'll have a look at it.

If you want to study how unicode can be handled in C for GEM, you can look into the Highwire sources. Also in GEMDict (see sig). As for your specific question, I can't help you; I've never used that function.
Jean-François
GEMDict – GEMClip

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Sun Dec 17, 2017 4:41 pm

Thank for all the suggestions. Here is what i found out so far:

- One of my problems was a wrong implementation of the binding, passing in an invalid mapping mode, which caused NVDI to always return -1.
- According to the NVDI documentation, unicode support was added (at least partially) in NVDI 4.
- vqt_char_index is supported by NVDI, but it is difficult to use:
- when using mapping 0->2 (direct to unicode), then it behaves differently when used on a Speedo font vs Truetype font. For speedo fonts,
the source index is expected to be a BICS character Index (as opposed to ID), which can be different between fonts. Even worse, that index might
be translated by that spdchar.map. When used on a Truetype font, it is just a glyph index. When trying to use it on a bitmap font, the results seem
to be totally unpredictable.
- despite the documentation, vst_map_mode always seems to return 1, even on invalid mapping modes

Conclusion: as it turns out, that function is rather useless, at least for my purposes. What i wanted to use it for, is to check whether a certain unicode character is available in the font. That seems to be not possible in a reliable way. I think i just use builtin translation tables for this purpose.

ThorstenOtto
Captain Atari
Captain Atari
Posts: 312
Joined: Sun Aug 03, 2014 5:54 pm

Re: NVDI & character mapping

Postby ThorstenOtto » Sat Dec 30, 2017 1:31 pm

Some more findings: the BICS character mapping, as defined in the Atari compendium, seems to be at least partly wrong. There are also some mappings available n the net (eg. https://en.wikipedia.org/wiki/Bitstream ... racter_Set), which list only the IDs. These are incomplete, and seems to be even partly based on the compendium, and therefore have the same errors.

Until X11 R6.8.2, speedo fonts where also supported in X servers on linux. They seem to use the glyph index, and not the ID, for the mapping, which only works with certain fonts that where freely available at that time. All of these fonts (similar to the ones supplied with SpeedoGDOS) have 564 glyphs defined.

In the meantime i wrote a tool that is able to display them, as posted in viewtopic.php?f=14&t=32878

I'm still interested in any document that describes the file format, the compendium only describes the header, but not the character data.


Social Media

     

Return to “Coding”

Who is online

Users browsing this forum: No registered users and 6 guests