Transfer binary file via Python 3 sockets

This is the place to post any code that you want to share with the community. Only completed scripts should be posted here.
Note: posts here are not necessarily endorsed by the community, and may represent amateur or even bad practices.

Transfer binary file via Python 3 sockets

Postby DennisT » Tue Sep 13, 2016 8:36 pm

With the help of some users here I was able to write a client/server that transfers a binary file via a socket connection. There are 2 connections for each transfer. The first connection is for sending status updates and various info (chat). These are small messages (less than 20 characters) and are bidirectional. The second connection is for transferring the file from the client to the host. This connection is unidirectional.
Basic logic:
Client connects to the server and sends the file name and size (in bytes)
Server allocates an available transfer port(starts looking @ 49152), opens the output file (using the passed name). Then sends "OK2Send:###' (### is the transfer port number) back to the client.
Client opens second connection on the transfer port and sends the file.
Server saves passed file and once fully received (uses the size in bytes passed from the client) sends a TRXOK back to the client.

There is error checking and debug capabilities (set DEBUG = 1 in Functions.py). This script takes about 40ms to transfer a 160KB file from a Raspbian RPi to a Ubuntu 14.04 VM.

Notes: In client.py & server.py I include the Functions.py file but that is pathless. You'll probably want to add a path, add it to your function file(s), move all the function into your program, or ?
client.py is basic, you will need to write your code for coming up with the file name.
The file name is pathless. If you want to save it to a folder other than the one server.py is running from you will need to add that (see RecvFile function in Functions.py)

If you have any questions or comments feel free to post. Keep in mind I'm new to Python so my code is probably "funky".

Functions.py:
Code: Select all
#!/usr/bin/python3
#-------------------------------------------------------------
import socket               # Import socket module
DEBUG = 0
CHATPORT = 49152
#-------------------------------------------------------------
#Open_Port: Opens a listen on the next available port
#   returns the socket object & port #
#NOTE:  You must do a accept call when you're ready to accept a connection.
#  We don't do it here for obvious reasons.
#-------------------------------------------------------------
def Open_Port ():
    global DEBUG
    if DEBUG == 1 : print ('Open_Port:Start')
    Port = 49151
    XSS = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # Create a socket object AF_INET = ipv4; stream : tpc
    XSS.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    XSS.settimeout(0.2)
    Fhost = ''
    if DEBUG == 1 : print ('Open_Port:Opening next available port')
    while True:
        Port += 1
        if Port > 65535 : Port = 49153      # if this happens, well, that's a lotta ports!
        try:
            XSS.bind((Fhost, Port))        # Bind to the port
        except socket.error:
            continue
        if DEBUG == 1 : print ('Open_Port:Setting Listen on port:',str(Port))
        XSS.listen(1)
        break
    if DEBUG == 1 : print ('Open_Port:Opened port ',str(Port))
    if DEBUG == 1 : print ('Open_Port:Complete')
    return XSS, Port
#-------------------------------------------------------------
#Chat_Recv: Recv data from client.
#ChatSS: The Socket object for chatting with client
#Return: Returns the data sent by the client
def Chat_Recv (ChatSS):
    global DEBUG
    if DEBUG == 1 : print ('Chat_Recv:Start-receiving chat data')
    TimerCount = 0
    SavTimeout = ChatSS.gettimeout()
    ChatSS.settimeout(0.1)
    Data = ''
    while True :                    #Recv Loop
        try:
            RawData = ChatSS.recv(1024)
        except socket.error:
            TimerCount += 1              #increment timer count
            if TimerCount >= 5 :         #waited 1/2 sec, no activity so error out
                if DEBUG == 1 : print ('Chat_Recv:receiving chat data timeout FAIL')
                break                   #exit Recv Loop
            continue
        Data = RawData.decode()
        break                   #exit Recv Loop
    ChatSS.settimeout(SavTimeout)
    if DEBUG == 1 : print ('Chat_Recv:received chat data:',Data)
    if DEBUG == 1 : print ('Chat_Recv:Complete')
    return Data
#-------------------------------------------------------------
#Chat_Send: Send data to client.
#ChatSS: The Socket object for chatting with client
#Data: String to send to client (<= 1024 chars, non binary)
def Chat_Send (ChatSS,Data):
    global DEBUG
    if DEBUG == 1 : print ('Chat_Send:sending chat:',Data)
    TimerCount = 0
    Ret = True
    SavTimeout = ChatSS.gettimeout()
    ChatSS.settimeout(0.1)
    while True :                    #Send Loop
        try:
            ChatSS.sendall(Data.encode())
        except socket.error:
            TimerCount += 1              #increment timer count
            if TimerCount >= 5 :         #waited 1/2 sec, no activity so error out
                if DEBUG == 1 : print ('Chat_Send:sending ',Data,' to client FAILED')
                Ret = False
                break                   #exit Send Loop
            continue
        break                   #exit Send Loop
    ChatSS.settimeout(SavTimeout)
    if DEBUG == 1 : print ('Chat_Send:Complete')
    return Ret
#-------------------------------------------------------------
#RcvFile - Opens output file named FileName.  Opens receiving
#   socket connection, notifies ChatSS it is ok to send, receives
#   file, saves it, closes receiving socket connection and file.
#FileName: The file name sans path.
#FileSize: The file size in bytes
#ChatSS: The Socket object for chatting with client
def RecvFile (FileName,FileSize,ChatSS) :
    global DEBUG
    Ret = 'OK'
    if DEBUG == 1 : print ('RecvFile:Start-Filename:',FileName,' FileSize:',str(FileSize))
    while True:
        try:
            if DEBUG == 1 : print ('RecvFile:Opening output file')
            WriteFile = open(FileName,'wb')  #open output file
        except:
            Ret = 'ERR_FILE_CANNOT_OPEN'
            if DEBUG == 1 : print ('RecvFile:Opening output file FAILED')
            break
        if DEBUG == 1 : print ('RecvFile:RecvFile opening recv socket')
        XSS, Port = Open_Port()
        if DEBUG == 1 : print ('RecvFile:sending client OK2Send & port #')
        Data = 'OK2Send:'+str(Port)
        Err = Chat_Send (ChatSS,Data)
        if DEBUG == 1 : print ('RecvFile:Waiting for client to open connection')
        try:
            XferSS, ClientAddr = XSS.accept()     # Establish connection with client
        except:
            if DEBUG == 1 : print ('RecvFile:Timeout waiting for client to connect to xfer file')
            Ret = 'ERR_CLIENT_FILE_XFER_CONNECT_TIMEOUT'
            break
        if Err == False : break
        TimerCount = 0
        RcvSize = 0
        while True:                          #receive file
            try:
                RawData = XferSS.recv(1024)
            except socket.error:
                if DEBUG == 1 : print ('RecvFile:Waiting on file sender')
                TimerCount += 1              #increment timer count
                if TimerCount >= 5 :         #waited 1/2 sec, no activity so error out
                    if DEBUG == 1 : print ('RecvFile:Receiving File timed out')
                    Ret = 'ERR_FILE_RECV_TIMEOUT'
                    break                   #exit recv loop
                continue
            TimerCount = 0
            RcvSize = RcvSize + len(RawData)
            if not RawData : break                  #exit recv loop
            WriteFile.write(RawData)        #write raw data to file
            if RcvSize >= FileSize :
                if DEBUG == 1 : print ('RecvFile:Received the full file size ')
                break                   #exit recv loop
        #==============================================================
        if RcvSize != FileSize :
            if DEBUG == 1 : print ('RecvFile:Receive file size wrong!')
            Ret = 'ERR_RCV_FILE_SIZE_BAD'
        if Ret == 'OK' :
            if DEBUG == 1 : print ('RecvFile:Telling Client Trxfer was OK')
            Data = 'TRXOK'
            Err = Chat_Send (ChatSS,Data)
        break
    #==============================================================
    WriteFile.close()
    if XferSS : XferSS.close()
    if DEBUG == 1 : print ('RecvFile:Complete')
    return Ret
#-----------------------------------------------------------------------------------
# send a file to DestHost.  destHost must be running server portion.
# We send the filename, wait for a OK2Send, then send the file, Then send EOF, then wait for a TRXOK.
def SendFile (ChatSS,DestHost,Port):
    import os
    global DEBUG
    if DEBUG == 1 : print ('SendFile:Start - DestHost:',DestHost,' FileName:',FileName,' Port:',str(Port))
    XferSS = socket.socket()         # Create a socket object
    XferSS.settimeout(1.0)           #1 second timeout
    TimerCount = 0
    Err = 'OK'                       #Default to OK results
    while True:                      #main loop
        if DEBUG == 1 and TimerCount == 0 : print ('SendFile:Connecting to ',DestHost,' on Port:',str(Port))
        try:                         #try to connect to server
            XferSS.connect((DestHost, Port))
        except:                      #could not connect
            TimerCount += 1          #increment timer count
            if TimerCount >= 5 :
                Err = 'ERR_SEND_FILE_HOST_TIMEOUT'
                break                #after 5 seconds break out of the main loop
            continue                 #try again
        if DEBUG == 1 : print ('SendFile:Connected, opening ',FileName)
        try:
            File = open(FileName,'rb')  #open file for reading
        except:
            Err = 'ERR_CANNOT_OPEN_FILE'
            if DEBUG == 1 : print ('SendFile:opening ',FileName,' FAILED')
            break
        if DEBUG == 1 : print ('SendFile:sending ',FileName)
        while True:                  #send the file
            Data = File.read(1024)
            if not Data:             #hit EOF, no more data
                break                #Exit file send loop
            XferSS.sendall(Data)
        if DEBUG == 1 : print ('SendFile:done sending ',FileName,' so closing it')
        File.close()
        Data = Chat_Recv (ChatSS)
        if Data != 'TRXOK' :
            Err = 'ERR_FILE_SEND_FAILED'
        break
    XferSS.close()
    if DEBUG == 1 : print ('SendFile:Complete')
    return Err
#-------------------------------------------------------------


client.py:
Code: Select all
#!/usr/bin/python3
#-------------------------------------------------------------
# Client
#-------------------------------------------------------------
#import Functions
exec(open('Functions.py').read())
import socket
import os
def ServerConn (DestHost,FileName) :
    global CHATPORT
    global DEBUG
    ChatSS = socket.socket()         # Create a socket object
    TimerCount = 0
    Err = 'OK'
    if DEBUG == 1 : print ('ServerConn:opening ChatSS')
    while True :
        try:                         #try to connect to server
            ChatSS.connect((DestHost, CHATPORT))
        except:                      #could not connect
            TimerCount += 1          #increment timer count
            if TimerCount >= 5 :
                Err = 'ERROR_1:NOCONNECT'
                if DEBUG == 1 : print ('ServerConn:ChatSS socket open failed')
                break                #after 5 seconds break out of the main loop
            continue
        Data = FileName.strip()+':'+str(os.stat(FileName).st_size)
        if DEBUG == 1 : print ('ServerConn:sending chat data:',Data)
        Ret = Chat_Send (ChatSS,Data)
        if Ret == False :
            Err = 'ERR_NO_HOST'
            if DEBUG == 1 : print ('ServerConn:send chat failed')
            break
        if DEBUG == 1 : print ('ServerConn:receiving chat data.')
        RawData = Chat_Recv (ChatSS)
        if ':' in RawData :
            Data = RawData.split(':')
        else:
            if DEBUG == 1 : print ('ServerConn:failed receive in OK2Send:Port format')
            Err = 'ERR_OK2Send bad format recd'
            break
        if DEBUG == 1 : print ('ServerConn:received:',Data)
        if len(Data) != 2 :
            Err = 'ERR_HOST_SILENT'
            if DEBUG == 1 : print ('ServerConn:failed receive of OK2Send + port')
            break
        Port = int(Data[1])
        if DEBUG == 1 : print ('ServerConn:sending file')
        Err = SendFile (ChatSS,DestHost,Port)
        if Err != 'OK': break
        break
    ChatSS.close()
    if DEBUG == 1 : print ('ServerConn:Complete')
    return Err
#
import time
if DEBUG == 1 : start_time = time.time()
DestHost = 'MyHost'
FileName ='SomeBinaryFile.jpg'
Err = ServerConn(DestHost,FileName)
if DEBUG == 1 :
    print ('Main:Result=',Err)
    elapsed_time = time.time() - start_time
    print ('Main:Elapsed Time:',str(elapsed_time),'Secs')


server.py:
Code: Select all
#!/usr/bin/python3
#-------------------------------------------------------------
#Server
#-------------------------------------------------------------
#import Functions
exec(open('Functions.py').read())

import socket               # Import socket module
from threading import Thread
#-------------------------------------------------------------
#ClientConn: function to accept data from a client connection and
#do things.  Initially receive a binary file.
def ClientConn (ChatSS) :
    global DEBUG
    if DEBUG == 1 : print ('ClientConn:Starting')
    if DEBUG == 1 : print ('ClientConn:Receiving file name & size')
    while True :                        #main loop
        RawData = Chat_Recv(ChatSS)            #get file name + size
        if ':' in RawData :
            Data = RawData.split(':')
        else:
            Data = RawData
            if DEBUG == 1 : print ('ClientConn:Received data:',Data,' should be in a FileName:FileSize format and it is not')
            break
        if DEBUG == 1 : print ('ClientConn:Received data:',Data)
        FileName = Data[0]
        FileSize = int(Data[1])
        if DEBUG == 1 : print ('ClientConn:Received name:',FileName,' and size:'+str(FileSize))
        Err = RecvFile(FileName,FileSize,ChatSS)            #receive the file
        break
    ChatSS.close()
    if DEBUG == 1 : print ('ClientConn:Complete')
    return                  #closes thread

#-------------------------------------------------------------
#  Server Main
global CHATPORT
global DEBUG
OpenSS = socket.socket(socket.AF_INET,socket.SOCK_STREAM) # Create a socket object AF_INET = ipv4; stream : tpc
OpenSS.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
port = CHATPORT                 # Reserve a port for your service.
host = ''
OpenSS.bind((host, port))        # Bind to the port
OpenSS.listen(5)                 # Now wait for client connections, backlog up to 5
while True:
    ChatSS, ClientAddr = OpenSS.accept()     # Establish connection with client, ClientSS is the socket stream of the client
    if DEBUG == 1 : print ('Main:Have a connection to process from:',ClientAddr)
    t = Thread(target=ClientConn,args=(ChatSS,)) #thread : accepts multiple clients at the same time
    t.start()
#-------------------------------------------------------------
DennisT
 
Posts: 21
Joined: Thu Apr 21, 2016 4:00 pm

Re: Transfer binary file via Python 3 sockets

Postby wavic » Wed Sep 14, 2016 8:31 pm

An idea...
Pass a md5 file hash from the server to the client and then check it agiain once the file is transfered. If md5sum is different client may ask for the file again.
wavic
 
Posts: 165
Joined: Wed May 25, 2016 8:51 pm

Re: Transfer binary file via Python 3 sockets

Postby Ofnuts » Wed Sep 14, 2016 10:22 pm

wavic wrote:An idea...
Pass a md5 file hash from the server to the client and then check it agiain once the file is transfered. If md5sum is different client may ask for the file again.

A good ol' CRC should be enough.
This forum has been moved to http://python-forum.io/. See you there.
User avatar
Ofnuts
 
Posts: 2659
Joined: Thu May 14, 2015 9:46 am
Location: Paris, France, EU, Earth, Solar system, Milky Way, Local Cluster, Universe #32987440940987

Re: Transfer binary file via Python 3 sockets

Postby wavic » Thu Sep 15, 2016 7:02 am

I didn't know what crc is for until now. But so many algorithms
wavic
 
Posts: 165
Joined: Wed May 25, 2016 8:51 pm


Return to Completed Scripts

Who is online

Users browsing this forum: Bing [Bot] and 3 guests

cron