scripts for processing stuff

Often I need to write something up quick to process a file, or process files in a directory and I'll want a GUI for these things so I can pass them along to other people to use. Here are two python scripts that I use as templates for picking either a file or a directory and then doing something with it. The object here isn't to have everything "correct" according to the latest design fashion or methodology. With this script I can import a python module that takes a file or directory name and does the processing I want, plug in the function name, pass the parameter and then move on with my life.

Pick a file and do something with it:

from tkinter import *
from tkinter.messagebox import *
from tkinter.filedialog import *

class FileProcessor(Frame):

# Object constructor
    def __init__(self, parent=None):
        Frame.__init__(self, parent)

        self.txtFile = Entry(parent)
        self.txtFile.place(x=6,y=64,width=375,height=24)
        self.txtFile.insert(0,'')

        self.btnPickFile = Button(parent,text='Pick File', command=self.btnPickFileClick)
        self.btnPickFile.place(x=6,y=20,width=96,height=24)

        self.btnRun = Button(parent,text='RUN', command=self.btnRunClick)
        self.btnRun.place(x=6,y=114,width=96,height=30)


# Methods (event handlers) of object
    def btnPickFileClick(self):
        print (self.txtFile.get())
        self.txtFile.delete(0,END)
        self.txtFile.insert(0,askopenfilename())

    def btnRunClick(self):
        fileName = self.txtFile.get()
        self.runFile(fileName)

    def runFile(self, fileName):
        print('code goes here to run when click: ' + fileName)
        showinfo('The file is',fileName)

# Method called if script is run directly
# instead of imported or used as a class
if __name__ == '__main__':
    root = Tk()
    root.title('Process File')
    myForm = FileProcessor(root)
    myForm.pack()
    root.geometry("423x156")
    root.minsize(423,156)
    root.maxsize(423,156)
    root.mainloop()

Pick a directory and do something wit it:

from tkinter import *
from tkinter.messagebox import *
from tkinter.filedialog import *

class DirectoryProcessor(Frame):

# Object constructor
    def __init__(self, parent=None):
        Frame.__init__(self, parent)

        self.txtDir = Entry(parent)
        self.txtDir.place(x=6,y=64,width=375,height=24)
        self.txtDir.insert(0,'')

        self.btnPickFile = Button(parent,text='Pick Directory', command=self.btnPickFileClick)
        self.btnPickFile.place(x=6,y=20,width=96,height=24)

        self.btnRun = Button(parent,text='RUN', command=self.btnRunClick)
        self.btnRun.place(x=6,y=114,width=96,height=30)


# Methods (event handlers) of object
    def btnPickFileClick(self):
        print (self.txtDir.get())
        self.txtDir.delete(0,END)
        self.txtDir.insert(0,askdirectory())

    def btnRunClick(self):
        dirName = self.txtDir.get()
        self.runDir(dirName)

    def runDir(self, directoryName):
        print('code goes here to run when click: ' + directoryName)
        showinfo('The directory is',directoryName)

# Method called if script is run directly
# instead of imported or used as a class
if __name__ == '__main__':
    root = Tk()
    root.title('Process Directory')
    myForm = DirectoryProcessor(root)
    myForm.pack()
    root.geometry("423x156")
    root.minsize(423,156)
    root.maxsize(423,156)
    root.mainloop()

Putting a string on a single line

Often times text is written on multiple lines for the sake of clarity. Sometimes these lines need to be consolidated to a single line. This happens to me a lot with SQL queries that end up needing to be in a config file somewhere. I'll write them out all pretty so they make sense (as much sense as some SQL can make...) and then I'll need to consolidate that down to one line. Here is a python one liner that does just that, given a string named mystr

" ".join([x.strip() for x in mystr.split("\n")])

Finding potential problems in CSV files

I have to periodically work with CSV files from a variety of sources and the problems with them are pretty well known. Here is a little python script I use that helps me find values that contain double quotes, which are often not properly escaped in the files I receive

flines = open(fn,"r").readlines()
currline = flines[1]

for l in flines:
    vals = l.split(",")
    for v in vals:
        if '"' in v.strip()[1:-1]:
            print v

Reviewing files from Subversion

Recently I had to deploy some updates to a website and wanted to get a list of all the files that had been updated to make sure I got everything. With recent versions of subversion and using python this turned out to be rather simple:

First, get a list of all the files that have been checked in and stick them in a text file.

C:\Utils\svn-win32-1.4.3\bin>svn diff --summarize -r902:966 svn://myServer/myRepository/myProject/trunk > c:\diff.txt

Then use the following lines of python to get the file names and sort them to make tracking them down easy

mylines = open("c:\\diff.txt" , "r").readlines()
myarr = [x.split(" ")[-1].strip() for x in mylines]
myarr.sort()

Now print out myarr, put it in its own file or do whatever you want with it to deploy or review your updates.

working with utf16 in python

I got a file from a client that was exported from SQL Server and was encoded as utf16. I needed to do some work on it. I had to google around a bit to find some help on handling the gobble-de-gook that I was seeing

>>> f = open("f:\\contact.txt","r")
>>> l = f.readline()
>>> l
'\xff\xfe6\x003\x003\x00D\x003\x00A\x008\.....\n'

Here is how to do it

>>> import codecs
>>> f = codecs.open("f:\\contact.txt", "r", "utf16")
>>> l = f.readline()
>>> l
u'633D3A84-3870-4A93-9755-000215260850,8568,NULL,Scooby,Shaggy,NULL,,NULL,mymail@address.com,1902-06-01 00:00:00.000,NULL\r\n'
>>>