Python Parsing Text to CSV.

This is the place for queries that don't fit in any of the other categories.

Python Parsing Text to CSV.

Postby IrishRaymondo » Wed Aug 14, 2013 1:48 pm

Hi All,

I've been trying to parse a text file into a csv file.
I can parse the text into the csv file.

[Test 1]
This is line one.
This is line two.

[Test 1]
This is line three.
This is line four.

And what im trying to do is in the csv file is get it to look like this
Name---------------Second-------------Third
[Test 1]--------This is line one.-----This is line two.
[Test 1]-------This is line three.-----This is line four.

So what I'm trying to do is when you come to a "[" bracket drop down a line.

Can any body help me with this.

Thank you in advance.
Last edited by IrishRaymondo on Mon Aug 19, 2013 1:22 pm, edited 1 time in total.
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby stranac » Wed Aug 14, 2013 1:50 pm

Show us some code.
What have you tried, how have you failed and/or what are you having trouble with?
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1144
Joined: Thu Feb 07, 2013 3:42 pm

Re: Python Parsing Text to CSV.

Postby IrishRaymondo » Wed Aug 14, 2013 2:17 pm

Hi,

I got this working now pretty much but have just one question left.

Code: Select all
import csv
import itertools

with open('test.txt', 'r') as in_file:
    stripped = (line.strip() for line in in_file)
    lines = (line for line in stripped if line)
    grouped = itertools.izip(*[lines] * 3)
    with open('mycsv.csv', 'w') as out_file:
        writer = csv.writer(out_file)
        writer.writerow(('Name', 'Second', 'Third'))
        writer.writerows(grouped)


My only problem now is that when I run this code it does what i want but there is a empty line between each.
So its like this.
Name---------------Second-------------Third

[Test 1]--------This is line one.-----This is line two.

[Test 1]-------This is line three.-----This is line four.
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby IrishRaymondo » Wed Aug 14, 2013 3:55 pm

Ok this is what im trying to do is delete the blank lines in the text file using this code and then writing it to another txt file.

Code: Select all
with open("test.txt", "r") as in_file:
        for line in in_file:
            cleanedLine = line.strip()
            with open('test_one.txt', 'w') as out_file:
                if cleanedLine:
                    out_file.write(cleanedLine)
                    print(cleanedLine)


I cant get it to write to the text file however.
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby micseydel » Wed Aug 14, 2013 4:20 pm

Inside the loop, you open the file in write mode, which deletes the existing contents. Move the file opening to outside the loop and you should be golden. Note two more things though: when you strip a line, you lose its newline character and so need to write that for each file, and you can combine the file opening lines
Code: Select all
with open("test.txt", "r") as in_file, open('test_one.txt', 'w') as out_file:
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1358
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Python Parsing Text to CSV.

Postby IrishRaymondo » Mon Aug 19, 2013 11:56 am

Hi,

I have it deleting the spaces how i want it but for some reason its still adding in the spaces when i convert the text file into a csv file.

Code: Select all
def Delete_Blank():
    print "Start Delete_Blank"
    file_Output = open('lint_one.log', 'w')
    with open("lint.log", "r") as in_file:
        for line in in_file:
            cleanedLine = line.strip()
            if cleanedLine:
                ouput = (cleanedLine) + "\n"
                file_Output.write(ouput)
                print (ouput)
    print "Finish Delete_Blank"
   
def Test_123():
    print "Start Test_123"
    with open('lint_one.log', 'r') as in_file:
        stripped = (line.strip() for line in in_file)
        lines = (line for line in stripped if line)
        grouped = itertools.izip(*[lines] * 3)
        with open('mycsv.csv', 'w') as out_file:
            writer = csv.writer(out_file)
            writer.writerow(('Rule', 'Violation', 'Path'))
            writer.writerows(grouped)
    print "Finish Test_123"
         
Delete_Blank()
Test_123()
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby IrishRaymondo » Mon Aug 19, 2013 12:38 pm

Just after doing this to see what i get.

Code: Select all
 with open('mycsv.csv', 'w') as out_file:
            writer = csv.writer(out_file)
            writer.writerow(('Rule', 'Violation', 'Path'))
            writer.writerow(('Rule1', 'Violation2', 'Path3'))
            #writer.writerows(grouped)


and im still getting a gap between them and im not sure why even if i stop the delete function and just write two lines to the csv there is a gap
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby stranac » Mon Aug 19, 2013 1:03 pm

Open the file in binary mode:
Code: Select all
with open('mycsv.csv', 'wb') as out_file:
    # ...
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1144
Joined: Thu Feb 07, 2013 3:42 pm

Re: Python Parsing Text to CSV.

Postby IrishRaymondo » Mon Aug 19, 2013 1:22 pm

Hi,

Thank you i tried loads of different things and never once taught about opening it in binary thank you so much :)
IrishRaymondo
 
Posts: 6
Joined: Wed Aug 14, 2013 1:30 pm

Re: Python Parsing Text to CSV.

Postby tnknepp » Mon Aug 19, 2013 1:30 pm

You can also set the lineterminator. csv defaults to opening csv.writer to 'excel' format, which has a terminator of '\r\n' (i.e. the good old carriage return, line feed).

Code: Select all
with open('mycsv.csv','w') as out_file:
    writer = csv.writer(out_file,lineterminator='\n')
    for r in range(5):
        writer.writerow( ('blah','blah','blah')
Python: 2.7 via Anaconda
Numpy: 1.7
Pandas: 0.11
OS: Windows 7
IDE: Spyder/IPython
User avatar
tnknepp
 
Posts: 122
Joined: Mon Mar 11, 2013 7:41 pm

Re: Python Parsing Text to CSV.

Postby stranac » Mon Aug 19, 2013 1:31 pm

IrishRaymondo wrote:Hi,

Thank you i tried loads of different things and never once taught about opening it in binary thank you so much :)

You should pay closer attention to the docs, especially when you're seeing something unexpected.
The docs for csv.writer() do say(as do the docs for csv.reader()):
If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1144
Joined: Thu Feb 07, 2013 3:42 pm


Return to General Coding Help

Who is online

Users browsing this forum: Baldyr, Google [Bot] and 3 guests