Scripting news

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, Greenious, spiny, Sorgelig, Moderator Team

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Scripting news

Postby Locutus73 » Sat Feb 02, 2019 5:57 pm

Scripting news:

===========================
Scripts_MiSTer:
- security_fixes.sh added
This script (which must be run in a SSH session) interviews the user, checking current Linux security status and asking (when needed) if the user wants a fix for:
1) changing root password
2) adding CA certificates
3) changing SSH host keys
4) disabling SSH daemon (with optional scripts for re enabling/disabling from OSD)
5) disabling FTP daemon (with optional scripts for re enabling/disabling from OSD)
6) disabling Samba daemon (with optional scripts for re enabling/disabling from OSD)
7) implementing a firewall (with ports opened/closed by previous daemon scripts)

- firewall_off.sh added
- firewall_on.sh added
- ftp_off.sh added
- ftp_on.sh added
- samba_off.sh added
- samba_on.sh added
- ssh_off.sh added
- ssh_on.sh added
Self explanatory scripts... please update them to Version 1.0.1 - 2019-02-02 - Remounting / as RW before altering /etc/init.d/ so the script actually works from OSD.

- gdrive.sh.inc
- gdrive_saves_download.sh
- gdrive_saves_upload.sh
These scripts upload and download saves directory to your Google Drive; they require working CA certificates (use security_fixes.sh first). I take no responsibility for any data loss or anything, use the scripts at your own risk.
===========================
https://github.com/MiSTer-devel/Scripts_MiSTer



===========================
CIFS_MiSTer: Version 1.7 - 2019-02-02
- The script temporarily modifies the firewalling rules for querying the CIFS Server name with NetBIOS when needed.
CIFS_MiSTer: Version 1.6 - 2019-02-02
- The script tries to download kernel modules (when needed) using SSL certificate verification.
===========================
https://github.com/MiSTer-devel/CIFS_MiSTer



===========================
Updater_script_MiSTer: Version 2.0 - 2019-02-02
- Added ALLOW_INSECURE_SSH option: "true" will check if SSL certificate verification (see https://curl.haxx.se/docs/sslcerts.html ) is working (CA certificates installed) and when it's working it will use this feature for safe curl HTTPS downloads, otherwise it will use --insecure option for disabling SSL certificate verification. If CA certificates aren't installed it's advised to install them (i.e. using security_fixes.sh). "false" will never use --insecure option and if CA certificates aren't installed any download will fail. The script will download and update the simple one line update.sh with a newer one (same ALLOW_INSECURE_SSH option) when needed.
===========================
https://github.com/MiSTer-devel/Updater_script_MiSTer



Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Sun Feb 03, 2019 5:11 pm

===========================
CIFS_MiSTer: Version 1.8 - 2019-02-03
- Added MOUNT_AT_BOOT option: "true" for automounting CIFS shares at boot time; it will create start/kill scripts in /etc/network/if-up.d and /etc/network/if-down.d.
===========================
https://github.com/MiSTer-devel/CIFS_MiSTer

P.S.: the /etc/network/if-up.d kill script is good for anyone having problems with rebooting while having CIFS shares mounted.

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Tue Feb 05, 2019 3:12 pm

===========================
CIFS_MiSTer: Version 2.0 - 2019-02-05
- Renamed from mount_cifs.sh and umount_cifs.sh to cifs_mount.sh and cifs_umount.sh for having them sequentially listed in alphabetical order.
===========================
https://github.com/MiSTer-devel/CIFS_MiSTer

===========================
Scripts_MiSTer: 2019-02-05
- gdrive_config_download.sh added
- gdrive_config_upload.sh added
- gdrive_saves_download.sh updated
- gdrive_saves_upload.sh updated
- rclone.sh.inc added
- rclone_config_downoad.sh added
- rclone_config_upload.sh added
- rclone_saves_downoad.sh added
- rclone_saves_upload.sh added
===========================
https://github.com/MiSTer-devel/Scripts_MiSTer

rclone scripts allow you to upload and download saves or config directory to the cloud storages supported by rclone:
* Amazon Drive
* Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio)
* Backblaze B2
* Box
* Dropbox
* FTP Connection
* Google Cloud Storage (this is not Google Drive)
* Google Drive
* Hubic
* JottaCloud
* Mega
* Microsoft Azure Blob Storage
* Microsoft OneDrive
* OpenDrive
* Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
* Pcloud
* QingCloud Object Storage
* SSH/SFTP Connection
* Webdav
* Yandex Disk
* http Connection
The scripts need a working SSL setup, so they basically need CA certificates to be installed in your MiSTer Linux; you can use security_fixes.sh for that, just the CA certificates fix is mandatory. rclone scripts need a rclone.conf file generated on your PC; the instructions are there: https://github.com/MiSTer-devel/Scripts ... /rclone.md.

gdrive scripts do the same, but they are limited to Google Drive and, differently from rclone ones, they don't use a MiSTer directory in your Google Drive; generally speaking I prefer rclone scripts, but they are slow with Google Drive if you don't use your own client_id (see https://rclone.org/drive/#making-your-own-client-id). gdrive scripts are faster with Google Drive.

Regards.

Locutus73

ewok
Atari User
Atari User
Posts: 31
Joined: Tue Jul 17, 2018 4:15 pm

Re: Scripting news

Postby ewok » Wed Feb 06, 2019 5:12 pm

Hi Locutus73,

Can you update Linux too with the use of one of the scripts?

Thanks!



I just noticed that this is done via de core updater. ARM version is now dated HPS190205

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Wed Feb 06, 2019 7:25 pm

ewok wrote:Can you update Linux too with the use of one of the scripts?

Yes if you use SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer" in your updater ini file, but it's only for the brave and lazy ones...
It's an experimental do it at your own risk procedure.
The official one in the Wiki is the recommended one.

Regards.

Locutus73
Last edited by Locutus73 on Wed Feb 06, 2019 7:27 pm, edited 1 time in total.

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Wed Feb 06, 2019 7:27 pm

===========================
Scripts_MiSTer: security_fixes.sh
- Version 1.2.1 - 2019-02-06 - Refined the check for standard root password.
- Version 1.2 - 2019-02-06 - Added security fix for Samba minimum allowed protocol.
- Version 1.1.1 - 2019-02-06 - Changed the name of the repository Kernel file to zImage_dtb_socfpga-4.5.
- Version 1.1 - 2019-02-06 - Checking current Kernel release is 4.5.0-socfpga-r1 before updating it for firewalling/iptables support.
===========================
https://github.com/MiSTer-devel/Scripts_MiSTer

Regards.

Locutus73

SegaMan
Atari maniac
Atari maniac
Posts: 83
Joined: Sat Oct 27, 2018 12:59 pm

Re: Scripting news

Postby SegaMan » Wed Feb 06, 2019 8:09 pm

I am a little bit confused :)

Added 2 arcade cores over FTP (Ghoust´n Goblins, Crazy Kong).

When running the update script this 2 cores will tagged as "NewCores".
All other arcade cores installed on my SD Card tagged as "Nothing to update".
Should the 2 cores i added not also tagged as "Nothing to update"?

Only setting on my update.ini is:
DOWNLOAD_NEW_CORES="fales"
But when adding cores by "hand" (FTP) they should not be listed as "NewCore" anymore i think :shrug:

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Wed Feb 06, 2019 8:18 pm

SegaMan wrote:I am a little bit confused :)

Added 2 arcade cores over FTP (Ghoust´n Goblins, Crazy Kong).

When running the update script this 2 cores will tagged as "NewCores".
All other arcade cores installed on my SD Card tagged as "Nothing to update".
Should the 2 cores i added not also tagged as "Nothing to update"?

Only setting on my update.ini is:
DOWNLOAD_NEW_CORES="fales"
But when adding cores by "hand" (FTP) they should not be listed as "NewCore" anymore i think :shrug:


Adding the cores manyally they shouldn't be listed as NewCore anymore if you preserved the original name structure, i.e. GnG_20190121.rbf when REMOVE_ARCADE_PREFIX="true" or Arcade-GnG_20190121.rbf when REMOVE_ARCADE_PREFIX="false". And obviously they must be in the right directory, i.e. /media/fat/_Arcade when CORE_CATEGORY_PATHS["arcade-cores"]="$BASE_PATH/_Arcade".

Regards.

Locutus73

SegaMan
Atari maniac
Atari maniac
Posts: 83
Joined: Sat Oct 27, 2018 12:59 pm

Re: Scripting news

Postby SegaMan » Wed Feb 06, 2019 9:34 pm

Ok i see.
Did not have the Arcade Prefix on every core name.

thx for help

Lightwave
Atari freak
Atari freak
Posts: 67
Joined: Sun Dec 16, 2018 8:18 pm

Re: Scripting news

Postby Lightwave » Wed Feb 06, 2019 9:57 pm

Locutus73 wrote:
ewok wrote:Can you update Linux too with the use of one of the scripts?

Yes if you use SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer" in your updater ini file, but it's only for the brave and lazy ones...
It's an experimental do it at your own risk procedure.
The official one in the Wiki is the recommended one.

Regards.

Locutus73


I have this line (uncommented) in my update.ini, but it didn't appear to grab the latest Linux version on update. It worked fine for me last time the kernel was updated.

Thanks for all your work on these scripts, very helpful!

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Wed Feb 06, 2019 11:18 pm

Lightwave wrote:I have this line (uncommented) in my update.ini, but it didn't appear to grab the latest Linux version on update. It worked fine for me last time the kernel was updated.

Thanks for all your work on these scripts, very helpful!


Can you tell me the contents of the /media/fat/#Scripts/.mister_updater/ directory?

Regards.

Locutus73

MattE888
Atari freak
Atari freak
Posts: 51
Joined: Fri Nov 02, 2018 5:34 pm

Re: Scripting news

Postby MattE888 » Thu Feb 07, 2019 1:27 pm

I haven't used the updater script in a few weeks. Does the updater script still just update cores, or does it now do these new things? If so, are these new things optional?

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Thu Feb 07, 2019 1:39 pm

MattE888 wrote:I haven't used the updater script in a few weeks. Does the updater script still just update cores, or does it now do these new things? If so, are these new things optional?

Everything is optional.
It’s all described here: https://github.com/MiSTer-devel/Updater_script_MiSTer
You can read the description of each option here: https://github.com/MiSTer-devel/Updater ... h#L46-L106

Regards.

Locutus73

MattE888
Atari freak
Atari freak
Posts: 51
Joined: Fri Nov 02, 2018 5:34 pm

Re: Scripting news

Postby MattE888 » Thu Feb 07, 2019 2:47 pm

Thanks for the clarification.

Lightwave
Atari freak
Atari freak
Posts: 67
Joined: Sun Dec 16, 2018 8:18 pm

Re: Scripting news

Postby Lightwave » Thu Feb 07, 2019 3:29 pm

Locutus73 wrote:
Lightwave wrote:I have this line (uncommented) in my update.ini, but it didn't appear to grab the latest Linux version on update. It worked fine for me last time the kernel was updated.

Thanks for all your work on these scripts, very helpful!


Can you tell me the contents of the /media/fat/#Scripts/.mister_updater/ directory?

Regards.

Locutus73


Sure, here are the file names:

menu_20190204.rbf
MiSTer_20190205
release_20190119.rar

I have since updated manually using the MiSTer SD card utility.

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Thu Feb 07, 2019 5:49 pm

Lightwave wrote:Sure, here are the file names:

menu_20190204.rbf
MiSTer_20190205
release_20190119.rar

I have since updated manually using the MiSTer SD card utility.


Perfect, can you post here the content of your INI?

Thank you in advance.

Locutus73

Lightwave
Atari freak
Atari freak
Posts: 67
Joined: Sun Dec 16, 2018 8:18 pm

Re: Scripting news

Postby Lightwave » Thu Feb 07, 2019 9:02 pm

Locutus73 wrote:
Lightwave wrote:Sure, here are the file names:

menu_20190204.rbf
MiSTer_20190205
release_20190119.rar

I have since updated manually using the MiSTer SD card utility.


Perfect, can you post here the content of your INI?

Thank you in advance.

Locutus73


Sure:

Code: Select all

CORE_CATEGORY_PATHS["cores"]="$BASE_PATH"
CORE_CATEGORY_PATHS["console-cores"]="$BASE_PATH"

REPOSITORIES_FILTER="Minimig Genesis NES SNES TurboGrafx16 arcade-cores"

; Uncomment below to update Linux
SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer"

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Thu Feb 07, 2019 10:29 pm

Lightwave wrote:Sure:

Code: Select all

CORE_CATEGORY_PATHS["cores"]="$BASE_PATH"
CORE_CATEGORY_PATHS["console-cores"]="$BASE_PATH"

REPOSITORIES_FILTER="Minimig Genesis NES SNES TurboGrafx16 arcade-cores"

; Uncomment below to update Linux
SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer"


I’m not at home or in the office now, so I can’t test it, but right off the bat, the first thing that comes to my mind is adding the SD-Installer to the REPOSITORIES-FILTER.
Let me know...

Regards.

Locutus73

Lightwave
Atari freak
Atari freak
Posts: 67
Joined: Sun Dec 16, 2018 8:18 pm

Re: Scripting news

Postby Lightwave » Thu Feb 07, 2019 11:25 pm

Locutus73 wrote:
Lightwave wrote:Sure:

Code: Select all

CORE_CATEGORY_PATHS["cores"]="$BASE_PATH"
CORE_CATEGORY_PATHS["console-cores"]="$BASE_PATH"

REPOSITORIES_FILTER="Minimig Genesis NES SNES TurboGrafx16 arcade-cores"

; Uncomment below to update Linux
SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer"


I’m not at home or in the office now, so I can’t test it, but right off the bat, the first thing that comes to my mind is adding the SD-Installer to the REPOSITORIES-FILTER.
Let me know...

Regards.

Locutus73


I don't think it's the repositories filter, as it worked before with it the same line...

BUT I think I just figured it out :) The comment prefix I added was a semicolon (used in MiSTer.ini), rather than a hash as used in your scripts.
I'm guessing the parser just choked on an unexpected char?

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Fri Feb 08, 2019 12:14 am

Lightwave wrote:BUT I think I just figured it out :) The comment prefix I added was a semicolon (used in MiSTer.ini), rather than a hash as used in your scripts.
I'm guessing the parser just choked on an unexpected char?


There’s no parser, and the INI isn’t really an ini... it’s a piece of bash script included in the main updater script just after the option variables are assigned with default values, so you can assign them again.
Standard bash code and comments.

Regards.

Locutus73

neumann
Atarian
Atarian
Posts: 5
Joined: Fri Nov 16, 2018 12:23 am

Re: Scripting news

Postby neumann » Thu Feb 21, 2019 9:32 am

Hi Locutus73,

thanks for all your awesome scripting!

If I understand your update script correctly, I can define in ini files user variables, which would basically overwrite the predefined values in your script. So if I like to add an additional repository to the predefined ones I need to repeat the repositories listed in your update script correct?

Or, is there any way of adding additional repositories to your predefined ones without overwriting?

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Thu Feb 21, 2019 11:27 am

neumann wrote:Hi Locutus73,

thanks for all your awesome scripting!

If I understand your update script correctly, I can define in ini files user variables, which would basically overwrite the predefined values in your script. So if I like to add an additional repository to the predefined ones I need to repeat the repositories listed in your update script correct?

Or, is there any way of adding additional repositories to your predefined ones without overwriting?


You're correct with the first assumption.
You have to repeat (or edit/modify) the repositories listed in the default config.

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Sat Mar 02, 2019 7:22 pm

===========================
Updater_script_MiSTer:
- updater.sh - Version 2.1 - 2019-02-23 - Code review by frederic-mahe, now the script is more standardized and elegant, thank you very much; ALLOW_INSECURE_SSH renamed to ALLOW_INSECURE_SSL.
- mister_updater.sh - Version 2.1 - 2019-03-02 - Linux updating now supports subdirectories under /media/fat/linux.
===========================
https://github.com/MiSTer-devel/Updater_script_MiSTer

===========================
Scripts_MiSTer:
- timezone.sh - Version 1.2 - 2019-03-02 - Changed "/media/fat/timezone" to "/media/fat/linux/timezone", removed -k option from curl.
===========================
https://github.com/MiSTer-devel/Scripts_MiSTer

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Sun Mar 03, 2019 12:16 pm

===========================
Updater_script_MiSTer:
- mister_updater.sh - Version 2.1.1 - 2019-03-03 - Improved date-time parsing for additional repositories; added MiSTer_MidiLink installation scripts to ADDITIONAL_REPOSITORIES.
===========================
https://github.com/MiSTer-devel/Updater_script_MiSTer

===========================
Scripts_MiSTer:
- change_ini_property.sh.inc - Version 1.1.1 - 2019-03-03 - Corrected a bug in MiSTer.ini download.
===========================
https://github.com/MiSTer-devel/Scripts_MiSTer

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 493
Joined: Wed Feb 07, 2018 6:13 pm

Re: Scripting news

Postby Locutus73 » Sat Apr 27, 2019 8:36 pm

===========================
Scripts_MiSTer: 2019-04-27
- wiimote_on.sh added
- wiimote_off.sh added
===========================
Nintendo Wiimote support as Lightgun and joypad with a standard Bluetooth USB dongle


https://github.com/MiSTer-devel/Scripts_MiSTer

Regards.

Locutus73



Return to “MiSTer”

Who is online

Users browsing this forum: uXe and 14 guests