[REQUEST] SD-Installer Option to Download Latest Cores

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, Greenious, spiny, Sorgelig, Moderator Team

JackBurton
Atari User
Atari User
Posts: 37
Joined: Thu Oct 04, 2018 3:55 am

[REQUEST] SD-Installer Option to Download Latest Cores

Postby JackBurton » Sun Feb 10, 2019 7:22 am

The SD-Installer has been working like a champ and I have very few complaints.

Would it be possible to have installer options to curl all cores, scripts, filters, and empty rom folders integrating the updater script? I've been setting up multiple DE10-nano systems back to back and it would be amazing if the installer could perform this step without having to do it through MiSTer.

Maybe this has already been discussed but I couldnt' find anything.

Thanks for the amazing development!

Locutus73
Captain Atari
Captain Atari
Posts: 489
Joined: Wed Feb 07, 2018 6:13 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Locutus73 » Sun Feb 10, 2019 9:22 am

Well, it’s not difficult to put together some mingw binaries and a batch script which can run the updater script. The updater would need some minimal changes (which can be patched live with a sed command), because it’s tailored to put some specific files (i.e. main MiSTer and menu.rbf) in a specific target directory (/media/fat).

Regards.

Locutus73

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5106
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Sorgelig » Sun Feb 10, 2019 10:54 am

actually it's enough to keep one SD card in MiSTer and update it regularly.
And then it can be cloned of copied to other SD cards when needed.

I don't see the needs of making SD-Installer more complicated.

JackBurton
Atari User
Atari User
Posts: 37
Joined: Thu Oct 04, 2018 3:55 am

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JackBurton » Sun Feb 10, 2019 6:23 pm

Sorgelig wrote:actually it's enough to keep one SD card in MiSTer and update it regularly.
And then it can be cloned of copied to other SD cards when needed.

I don't see the needs of making SD-Installer more complicated.


Fair enough, though cloning my main SD card means that I have to manually remove all legally obtained games before copying.

If I keep a secondary blank SD card for the purpose of cloning, the target SD card would have to be the same size or bigger than the source SD card, and it would copy all of the empy bits as well which would take extra time and space, right? I'm assuming your thinking of using something like dd or a GUI variant of it.

What I'm doing now is updating one card, manually copying the files to the computer and then copying those to each card that has SD-Installer run. I like to keep it up to date so I have to repeat the process each time I prepare a card.

I wondered if it could be as simple as a few extra checkboxes on the SD-Installer GUI: "Download latest cores" and "Download config/filter files"

Thanks!

JackBurton
Atari User
Atari User
Posts: 37
Joined: Thu Oct 04, 2018 3:55 am

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JackBurton » Sun Feb 10, 2019 6:27 pm

Locutus73 wrote:Well, it’s not difficult to put together some mingw binaries and a batch script which can run the updater script. The updater would need some minimal changes (which can be patched live with a sed command), because it’s tailored to put some specific files (i.e. main MiSTer and menu.rbf) in a specific target directory (/media/fat).

Regards.

Locutus73


A separate executable would work great, as well. I (possibly incorrectly) assumed that almost everyone is going to want all of the official cores since they take up minimal space. I also thought that the SD-Installer already knows which drive is the one you want to drop the cores onto because it was responsible for formatting it. No reason a secondary executible couldn't have another GUI to do the same thing but it seemed like extra effort.

But I'm not much of a programmer so I also respect the decision of those who are able to write the code.

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 815
Joined: Mon Nov 04, 2013 5:23 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JimDrew » Sun Feb 10, 2019 7:24 pm

People (by the dozens) have asked me if there will ever be any Windows or Mac (non-Linux) native application that can download the latest cores and update the SD card. Nobody likes to deal with Linux.
I am the flux ninja

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5106
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Sorgelig » Sun Feb 10, 2019 7:49 pm

JimDrew wrote:People (by the dozens) have asked me if there will ever be any Windows or Mac (non-Linux) native application that can download the latest cores and update the SD card. Nobody likes to deal with Linux.

Updates script for MiSTer does its job equally for all users.

Locutus73
Captain Atari
Captain Atari
Posts: 489
Joined: Wed Feb 07, 2018 6:13 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Locutus73 » Mon Feb 11, 2019 9:43 am

JackBurton wrote:A separate executable would work great, as well.

JimDrew wrote:People (by the dozens) have asked me if there will ever be any Windows or Mac (non-Linux) native application that can download the latest cores and update the SD card. Nobody likes to deal with Linux.

Sorgelig wrote:Updates script for MiSTer does its job equally for all users.

My point is that reinventing the wheel is useless: normal MiSTer users can use the updater script through the main menu interface, no dealing with Linux, just click, click, click… but if you want to serialize the build of multiple MiSTer systems in order to sell them, the updater is a Bash script and Bash is ubiquitous; it natively runs on all Unix systems (OSX is Unix) and you can easily use it under Windows with different methods. i.e. you can install Cygwin https://www.cygwin.com/, open a Cygwin Terminal, go into your SD (i.e. if it’s E:, just type cd /cygdrive/e) and then run

Code: Select all

curl -sLf https://github.com/MiSTer-devel/Updater_script_MiSTer/blob/master/mister_updater.sh?raw=true | sed "s/\/media\/fat/$(pwd | sed 's/\//\\\//g')/g" | sed 's/AUTOREBOOT="true"/AUTOREBOOT="false"/g' | bash -

Obviously the #Scripts directory will lack the update.sh script, but guess what? You can get it with another curl...
Making money selling open source stuff is perfectly legit, but please do your homework and share the results.

Regards.

Locutus73

JackBurton
Atari User
Atari User
Posts: 37
Joined: Thu Oct 04, 2018 3:55 am

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JackBurton » Mon Feb 11, 2019 7:32 pm

I didn't mean to stir up controversy and will happily use the script, once I install cygwin. Thanks!

Locutus73
Captain Atari
Captain Atari
Posts: 489
Joined: Wed Feb 07, 2018 6:13 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Locutus73 » Tue Feb 12, 2019 8:32 am

JackBurton wrote:I didn't mean to stir up controversy and will happily use the script, once I install cygwin. Thanks!


No controversy at all, I was just saying... and please share any findings.

Regards.

Locutus73

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 815
Joined: Mon Nov 04, 2013 5:23 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JimDrew » Tue Feb 12, 2019 8:22 pm

Perhaps I should be more clear about what people have requested. Nobody but hard core people ever want to deliberately use a command line interface for anything. We live in modern times and everyone wants a GUI where you can just click a button and magically every core is updated on the SD card, with that card being updated in a SD card reader connected to a PC, not the DE-10/MiSTer setup. So... a PC (and/or Mac) program that will go to the github site, get the latest release of each core, compare to what is on the SD card currently and then update it as necessary (with an option to delete the older version).
I am the flux ninja

rjbrown99
Atarian
Atarian
Posts: 6
Joined: Sat Feb 27, 2016 7:38 am

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby rjbrown99 » Wed Feb 13, 2019 3:28 am

A quote from the MacOS SD card creation script: https://github.com/MiSTer-devel/Updater_script_MiSTer

"Simply put update.sh in your SD and launch it through MiSTer main menu OSD (press F12 and then Scripts)."

Doesn't that mean you can run it from the main menu/OSD?

ericgus
Captain Atari
Captain Atari
Posts: 456
Joined: Fri Apr 08, 2016 4:53 am
Location: Boston MA - USA

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby ericgus » Wed Feb 13, 2019 4:58 am

JimDrew wrote:Perhaps I should be more clear about what people have requested. Nobody but hard core people ever want to deliberately use a command line interface for anything. We live in modern times and everyone wants a GUI where you can just click a button and magically every core is updated on the SD card, with that card being updated in a SD card reader connected to a PC, not the DE-10/MiSTer setup. So... a PC (and/or Mac) program that will go to the github site, get the latest release of each core, compare to what is on the SD card currently and then update it as necessary (with an option to delete the older version).



You can do this already with the F12 main mister menu, go to scripts, then select "updater" if you have put the updater.sh script on your mister SD card.. nothing to do .. just sit back and watch it automatically update/install any missing, new or updated cores, filters, etc.. . its literally just pressing the F12 key and hitting enter on the "updater" script.. its pretty brain dead simple. No need to remove your SD card from the mister, just when you feel like you want to see if there are any updates, from the main menu again hit f12, select SCRIPTS, go to updater and re-run it right from the mister's menu core onscreen UI.. pretty simple. No intermediate PC required, no SD card reader needed for updates/new cores, no SD card removal/swapping none of that.
Last edited by ericgus on Wed Feb 13, 2019 5:02 am, edited 4 times in total.

ericgus
Captain Atari
Captain Atari
Posts: 456
Joined: Fri Apr 08, 2016 4:53 am
Location: Boston MA - USA

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby ericgus » Wed Feb 13, 2019 5:00 am

rjbrown99 wrote:A quote from the MacOS SD card creation script: https://github.com/MiSTer-devel/Updater_script_MiSTer

"Simply put update.sh in your SD and launch it through MiSTer main menu OSD (press F12 and then Scripts)."

Doesn't that mean you can run it from the main menu/OSD?



Yes..

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 815
Joined: Mon Nov 04, 2013 5:23 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JimDrew » Wed Feb 13, 2019 6:46 am

ericgus wrote:You can do this already with the F12 main mister menu, go to scripts, then select "updater" if you have put the updater.sh script on your mister SD card.. nothing to do .. just sit back and watch it automatically update/install any missing, new or updated cores, filters, etc.. . its literally just pressing the F12 key and hitting enter on the "updater" script.. its pretty brain dead simple. No need to remove your SD card from the mister, just when you feel like you want to see if there are any updates, from the main menu again hit f12, select SCRIPTS, go to updater and re-run it right from the mister's menu core onscreen UI.. pretty simple. No intermediate PC required, no SD card reader needed for updates/new cores, no SD card removal/swapping none of that.


How can the script update the SD card while the SD card is in a PC's SD card reader? - It can't. Furthermore, having the DE-10 without an internet connection (Wifi or Ethernet) is not going to work. I think the people that are involved with the MiSTer project are sometimes "too close" to the project to objectively look at it as someone just discovering it, and so there are many assumptions made (like an internet connection).

Having a SD card plugged into a PC (or Mac) and running a GUI based program where the user can instruct the program what drive letter the SD card is assigned to, is what people are after. This eliminates someone from having to locate a script file and actually copy it on to the SD card, setup up the internet on the DE-10 (meaning WiFi or Ethernet), and then run the script. The MiSTer installer program is great because it does all of the work, and it's a GUI based application. Something similar for updates would be handy for those that do not have any knowledge about script files - which is most of the general public.
I am the flux ninja

warham
Atari maniac
Atari maniac
Posts: 90
Joined: Fri Oct 12, 2018 7:17 am

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby warham » Wed Feb 13, 2019 7:08 am

JimDrew wrote:
ericgus wrote:You can do this already with the F12 main mister menu, go to scripts, then select "updater" if you have put the updater.sh script on your mister SD card.. nothing to do .. just sit back and watch it automatically update/install any missing, new or updated cores, filters, etc.. . its literally just pressing the F12 key and hitting enter on the "updater" script.. its pretty brain dead simple. No need to remove your SD card from the mister, just when you feel like you want to see if there are any updates, from the main menu again hit f12, select SCRIPTS, go to updater and re-run it right from the mister's menu core onscreen UI.. pretty simple. No intermediate PC required, no SD card reader needed for updates/new cores, no SD card removal/swapping none of that.


How can the script update the SD card while the SD card is in a PC's SD card reader? - It can't. Furthermore, having the DE-10 without an internet connection (Wifi or Ethernet) is not going to work. I think the people that are involved with the MiSTer project are sometimes "too close" to the project to objectively look at it as someone just discovering it, and so there are many assumptions made (like an internet connection).


Having a SD card plugged into a PC (or Mac) and running a GUI based program where the user can instruct the program what drive letter the SD card is assigned to, is what people are after. This eliminates someone from having to locate a script file and actually copy it on to the SD card, setup up the internet on the DE-10 (meaning WiFi or Ethernet), and then run the script. The MiSTer installer program is great because it does all of the work, and it's a GUI based application. Something similar for updates would be handy for those that do not have any knowledge about script files - which is most of the general public.



Ill reply to this as I know nothing about scripting. Yes, internet connection is assumed.

There are continuous updates and people probably dont want to keep taking the sd card out of the DE-10, i dont. Just run the updater from the DE-10 OSD once in a while and your good to go. I assume the update.sh script gets installed with the SD installer now?
If not then yah it needs to be copied to the card but so do the arcade roms. Its a hobbyist project.

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 815
Joined: Mon Nov 04, 2013 5:23 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JimDrew » Wed Feb 13, 2019 7:07 pm

warham wrote:Its a hobbyist project.


It was at one time, but now with literally thousands of add-on boards sold, it is far from just a simple hobbyist project. It could grow exponentially, but these small details are what is holding it back.

People DO want to remove the SD card and update it using their PC. This is the biggest request that I have had, and not just by a few people - like hundreds!
Last edited by JimDrew on Thu Feb 14, 2019 5:13 am, edited 1 time in total.
I am the flux ninja

ericgus
Captain Atari
Captain Atari
Posts: 456
Joined: Fri Apr 08, 2016 4:53 am
Location: Boston MA - USA

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby ericgus » Wed Feb 13, 2019 7:38 pm

JimDrew wrote:How can the script update the SD card while the SD card is in a PC's SD card reader? -


You dont.

The point is you leave it in the Mister and have a network connection into the mister.

everything is run directly from the mister.. you dont ever really need to remove the SD card from the Mister to do updates or add new cores..

for roms/etc you can simply use an SFTP client to move any dsk, img, rom files over required by the core if you need to.. but in terms of updating the cores themselves no, you dont really need to use a PC with an SD card reader as long as you have the update.sh script and an internet connection on your mister.. failing those two things then yea you would need to manually manage all that mess yourself. But the point is, there is no reason to need to do that now with the update.sh script and the ability to run unix scripts directly from within the mister's main menu under F12 .. (and having your mister plugged into the network either rj45 or usb wifi).

ericgus
Captain Atari
Captain Atari
Posts: 456
Joined: Fri Apr 08, 2016 4:53 am
Location: Boston MA - USA

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby ericgus » Wed Feb 13, 2019 7:44 pm

JimDrew wrote:
People DO want to remove the SD card and update it using their PC. This is the biggest request that I have had, and not just by a few people - like hundreds!


While I suppose they can, You are better off to tell them to copy the update.sh script over to their mister SD card after they have done the initial Mister SD card prep setup and connect their mister to the internet with network cable (or wifi usb) .. from there thats all they need to do .. and run the update script from the misters menu core via F12 .. its really that easy. If cores etc are not on the SD card, update.sh will install them, and update them from github directly going forward on subsequent runs.

User avatar
Eero Tamminen
Atari God
Atari God
Posts: 1965
Joined: Sun Jul 31, 2011 1:11 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Eero Tamminen » Wed Feb 13, 2019 9:51 pm

JimDrew wrote:People DO want to remove the SD card and update it using their PC. This is the biggest request that I have had, and not just by a few people - like hundreds!


If there are so many people wanting it, why they just don't do it? How they can expect people who don't e.g. have Windows and no interest on using / developing / testing stuff on it, to do stuff for them?

(For example only times I use Windows are when my retired relatives have some problem with it, or when I'm forced at work to use one because some incompetent in our IT / sub-contracting has failed to follow industry standards for Web development / testing, and produced another Windows specific internal web site. Doesn't exactly endear me to it.)

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 815
Joined: Mon Nov 04, 2013 5:23 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby JimDrew » Wed Feb 13, 2019 10:21 pm

ericgus wrote:
JimDrew wrote:
People DO want to remove the SD card and update it using their PC. This is the biggest request that I have had, and not just by a few people - like hundreds!


While I suppose they can, You are better off to tell them to copy the update.sh script over to their mister SD card after they have done the initial Mister SD card prep setup and connect their mister to the internet with network cable (or wifi usb) .. from there thats all they need to do .. and run the update script from the misters menu core via F12 .. its really that easy. If cores etc are not on the SD card, update.sh will install them, and update them from github directly going forward on subsequent runs.


The problem is THAT is a LOT of work to figure out for the general public, with the mass majority of the people not having any idea what a script file is, and have never even heard of Linux or ROMs. THAT is the reality of the majority of people I have dealt with. Unless it is something that they can physically see - like drag and drop copying, or selecting files from a requestor, they are lost and I would say that virtually nobody is connecting their DE-10/MiSTer via Ethernet or WiFi.
I am the flux ninja

Locutus73
Captain Atari
Captain Atari
Posts: 489
Joined: Wed Feb 07, 2018 6:13 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Locutus73 » Wed Feb 13, 2019 10:34 pm

An easy approach could be:
a) including update.sh with the SD-Installer
b) including the necessary MinGW binaries (bash, cURL, sed, etc.) and cooking a simple update.bat for running the upadater in Windows with a double click.

It’s pretty easy, I already did that for other scripts I made, the only thing that needs some study is the licensing terms for including MinGW stuff.

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 489
Joined: Wed Feb 07, 2018 6:13 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby Locutus73 » Wed Feb 13, 2019 10:53 pm

JimDrew wrote:The problem is THAT is a LOT of work to figure out for the general public, with the mass majority of the people not having any idea what a script file is, and have never even heard of Linux or ROMs. THAT is the reality of the majority of people I have dealt with. Unless it is something that they can physically see - like drag and drop copying, or selecting files from a requestor, they are lost and I would say that virtually nobody is connecting their DE-10/MiSTer via Ethernet or WiFi.

I think you’re over exaggerating things in order to stick to your point: what should do with MiSTer anyone who never heard about ROMs???
Honestly it’s not a general public product, it’s more a tinkerer project and I don’t think virtually no MiSTer system is networked.

Regards.

Locutus73

djmartins
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 104
Joined: Wed Nov 21, 2018 10:26 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby djmartins » Thu Feb 14, 2019 12:52 am

JimDrew wrote:The problem is THAT is a LOT of work to figure out for the general public, with the mass majority of the people not having any idea what a script file is, and have never even heard of Linux or ROMs. THAT is the reality of the majority of people I have dealt with. Unless it is something that they can physically see - like drag and drop copying, or selecting files from a requestor, they are lost and I would say that virtually nobody is connecting their DE-10/MiSTer via Ethernet or WiFi.


I'd bet a lot of MiSTers are networked and it is not a project for those who can't perform basic computer stuff such as search for roms and run a simple premade script on MiSTer.
It's a complicated machine based on a loss leader FPGA board with addons and lot of fairly complex software that frankly has been made really easy to
work with but is by it's nature complex.
Arcade games, russian PDP-11 desktops and even a PDP-1 along with most major retro systems and consoles.
If you are helping people who can't with MiSTer that is great but the update script works fantastic.

hernan43
Retro freak
Retro freak
Posts: 10
Joined: Thu Dec 27, 2018 7:10 pm

Re: [REQUEST] SD-Installer Option to Download Latest Cores

Postby hernan43 » Thu Feb 14, 2019 1:49 am

Would it be possible to have some kind of SD card image that could be fed into something like Etcher? I’m thinking something that has the bare essentials and the updater script.

It wouldn’t need to constantly be kept up to date because the updater script would handle that.

I honestly can’t think of a more convenient update mechanism than the updater script. It seems like constantly updating the MiSTer by removing the card from the De10 and sticking it in your PC would be a nuisance and prone to unintentional screw ups.

— Ray


Return to “MiSTer”

Who is online

Users browsing this forum: solskogen and 7 guests