When authentication by keys worked fine

The data were supposed to be downloaded from several servers. The servers were not configured in the same way. One of them supported authentication using Public key and another not. Therefore the script had to take into account this fact. The environment on the client server, to whom the data should be downloaded to was quite tightened. No option to install any 3rd party application, that would allow to connect and authenticate to the SFTP server was allowed. Therefore, no Expect, no LFTP, or any other clients. 

SFTP allowing Public key authentication

That was quite easy and quick. I simply generated the pair of Private / Public key. As the job is running during the night, there is no password used in the keys. The Public key was sent to the administrator of the SFTP server, and I kept mine on the server. Several times, I just finetuned some loops and everything started to work swiftly without any issues.

Situation when authentication by keys is not supported

Another server unfortunately did not supported this method of authentication. because of that, there was required to perform another solution.  Because of tightened security, no mean to install anything else than core of the Operating System was possible. I googled a lot, and I finally found a workaround.

SFTP blocking Public key authentication

Indeed, it is not as secure as the first option, but it works. The download of the data by own job is possible if you use e.g. CURL instead of unsupported keys.


#purpose: script for downloading of files from SFTP using SSH public keys or using CURL if source server does not support SSH public/private keys combination

YdayYMD = $(date  --date="yesterday" +"%Y%m%d")                                  #date format like 20221115
YdayDM = $(date  --date="yesterday" +"%d%m")                                          #date format like 1511
Now = $(date  --date "today" +"%d. %m. %Y %H:%M:%S")                           #date time format like 15. 11. 2022 19:09:13
Day = $(date  --date="yesterday" +%u)                                                           #1 - for Monday (depends on operation system configuration)
LFridayYMD = $(date -d 'last Friday' +'%Y%m%d')                                        #20221028 - for last friday in format YYYYMMDD
LFridayDM = $(date -d 'last Friday' +'%d%m')                                                #2810 - for last friday in format DDMM
LFolder = "/johndoe/admin621/data/indices/"                                            #local folder for download of the files from SFTP servers
LogFile = "/​johndoe/admin621/data/indices/log.txt"                                 #text log files for info about downloaded files
MailRecipient = "distributionlist@domain.com"                                            #distribution group - recipients of the report and attached files
#MailRecipient="firstname.surname@​domain.com"                                 #e-mail address for development/debugging of the script
MailRecipientC = "firstname.surname@domain.com"                                 #recipient to CC of the email message
FTP1Login = "JohnDoe987"                                                                              #Username of the source​ SFTP server (CURL usage)
FTP1Pass = "789EodNhoj"                                                                                #Password of the source SFTP server (CURL usage)
FTP1Port =22                                                                                                     #TCP port - specified usage only, if the SFTP listens on non-standard port
#​FTP1="${​FTP1Login}@data.server1.com"                                                 #for use once the SFTP public key will work as expected
FTP1 ="data.server1.com"                                                                                 #FQDN of the SFTP server
FTP2Login = "MelodicJerry//23"                                                                      #Username of the source SFTP server
FTP2Port =22                                                                                                     #TCP port - specified usage only, if the SFTP listens on non-standard port
FTP2 = "${​FTP2Login}@subdomain.domain.tld"                                          #filling the server login for further use
FTPs = ([0]="${​FTP1}" [1]="${​FTP2}")                                                              #filling the array with the list of servers, the script will connect to
FTPsLen = ${#FTPs[@]}                                                                                    #SFTPs array length
FTP1Files = (/data/eu/100a${YdayDM}.csv
                                                                    #Filling the array with filenames to download in common days - yesterday timestamp in filename