Encoding / decoding / recoding strings - the mystery of carriage returns and new lines


#1

I’m running into what is probably a simple/silly problem, but after spending some time on various forums, it’s time to throw my hands in the air and ask for help.

I’m using ofxCsv in order to work with a CSV file in openFrameworks. The addon stores the CSV in a nested string vector like this vector<vector<string> > data. That seems easy enough. But when I pull that data, even though it is a string, it is being encoded / rendered differently than if I were to save the same text into a string testString. Here is what I mean.

When I do this:

    string stringTest = "one\ntwo\nthree";
    cout << "myString is " << stringTest << endl;
    
    string stringFromCsvDataVector = csv.data[1][9];
    cout << "stringFromCsvDataVector is " << stringFromCsvDataVector << endl;

This is what prints to the console:

stringTest is one
two
three
stringFromCsvDataVector is one\ntwo\nthree

When I add breakpoints to the lines containing cout above and debug in Xcode, here are the descriptions of the two vars that are returning:

Printing description of stringFromCsvDataVector:
(std::string) stringFromCsvDataVector = “U\xb4\x0f\xac\xd0\x10\xc1\xea\x10\x89E\xc8\x89U\xcc\x0f\xb7E\xc8\x0f\xb7U\xcc\x89E\xc8\x89U\xcc\x8bE\xc0\x8bU\xc4\x03E\xc8\x13U\xcc\x89E\xb0\x89U\xb4\x8bE\xb0\x8bU\xb4\x89\xc6\x89\xd7\x0f\xac\xfe\x10\xc1\xef\x10\x89\xf0%\xff\xff”

Printing description of stringTest:
(std::string) stringTest = “one\ntwo\nthree”

I want the \n embedded in a block of text in my CSV data file to be recognized as a new line when storing that block of text in a buffer that I’m writing to an ofFile, which is going to be turned into an .md file to make the CSV data readable. I’m sure there is an easy answer to this - but I can’t seem to find it.

Thanks in advance!


#2

Typically \n in a CSV file means the end of a row, so if you embed a \n in your csv, it will try to create a new row. If you want to include a new line, then you need to make sure your csv entry is wrapped in quotes and then you need to make sure that ofxCsv respects that standard.

See this for more: http://stackoverflow.com/questions/566052/can-you-encode-cr-lf-in-into-csv-files


#3

Yes - this is precisely the problem I’m trying to overcome! I believe that
ofxCsv cannot handle the new lines in my source csv, so I was going to
strip the new lines and the quotation marks, and put them back into the
output in the markdown file myself. However, I am unable to do that because
of the encoding/decoding described in the original post. Any thoughts on a
work around?


#4

Is your code on github? I think I need more code to help with this one …


#5

Thanks, @bakercp. Trying a different approach and reworking the code - will keep you posted!


#6

I’m actually also parsing CSV in oF today, so I’ll let you know if I come across any good solutions.


#7

Sounds good! If it’s any help, here is roughly where I’m at–

As far as I can tell, ofxCsv can’t handle line breaks in a CSV file, even if they are enclosed in " ". Google sheets allows you to easily find \n's and replace them, so I was thinking of replacing all the new lines with a key word (NEW_LINE), then using RegEx to replace those keywords with actual \n's when I output to my markdown file.


#8

That seems perfectly reasonable.


#9

Hi,

I’d like to add you to my professional network on LinkedIn.

  • Abraham

Confirm that you know Abraham: https://www.linkedin.com/e/v2?e=-z3hlkv-ibczdao8-d&a=preRegInvite&tracking=eml-guest-invite-cta&ek=invite_guest&sharedKey=JjcI8l1M&invitationID=6020019756692705281

You received an invitation to connect. LinkedIn will use your email address to make suggestions to our members in features like People You May Know. Unsubscribe here: https://www.linkedin.com/e/v2?e=-z3hlkv-ibczdao8-d&t=lun&midToken=AQFmrFVhGEd36Q&ek=invite_guest&loid=AQFNDgjzjEH_1AAAAU4toLg5DetdTZsnNslWQ1Gr8uIdgbeS2vTSTXq-gJCbskHqvJ9CN6JVa-Xrt1G5e0OQTl3GVRoSQYOuhSfiayqmmu-75YTNUg&eid=-z3hlkv-ibczdao8-d Learn why we included this at the following link: https://www.linkedin.com/e/v2?e=-z3hlkv-ibczdao8-d&a=customerServiceUrl&ek=invite_guest&articleId=4788
© 2015, LinkedIn Corporation. 2029 Stierlin Ct. Mountain View, CA 94043, USA


#10

Hi @abrahamavnisan . Sounds good. :wink:


#11

Ooops! Did not mean to do that :blush:

Though I do consider talk.olab an important part of my professional network!


#12

Abraham Avnisan would like to connect on LinkedIn. How would you like to respond?

Accept: https://www.linkedin.com/blink?simpleRedirect=0Ne38Rc3sOejoSdjsVcj0Mcz0Sfkh9rCZFt65QqnpKqipdcmMUimdGizRVpkJApn9xq7cCt7xBt5ZQs6lzoS4ZsTgCrT9Bq3RJ9zcZqmMCcj1vsClArCBJpn9vt7dBtmtvt7dOqmoZqSkCpnhFtCV9pSlipn9Mfm4CojcJczBzrnFHoCAJtCJIq3dWbjRBfP9SbSkLrmZzbCVFp6lHrCBIbDtTtOYLeDdMt7hE&msgID=I6020019772513619968_500&markAsRead=

View Abraham Avnisan's profile: https://www.linkedin.com/blink?simpleRedirect=34Vc38Ne3oRcPR4in9BoCRBripQu6lQnSlIqmpLsD0ZsTgCrT9Bq3RJ9zgZqmMCcj1vsClArCBJpn9vt7dBtmtvt7dOqmoZqSkCtD1KfngCojcJczBzrnFHoCAJtCJIq3dWbjRBfP9SbSkLrmZzbCVFp6lHrCBIbDtTtOYLeDdMt7hE&msgID=I6020019772513619968_500&markAsRead=

You are receiving Reminder emails for pending invitations. Unsubscribe here: https://www.linkedin.com/blink?simpleRedirect=rSB5cylyomNLc3gBoz1yd6kPdPxxd6oMe3oVpmlypCcRdjpCp34Od64PoPp2cylHr65QfmNFomRB9z0Sc30OfmhF9z4UczkMdP8VdzoRdPANc30Oc3oZp6BD9z4MnT9Bp6VFrmlOnThPpnlDnThPsCBCfmJB9DhUpnhvoDlPrDkJdiRytndKtiQNc5ZOpmhKqmRBsBZQsSlRpRZQsT9FpyRIrmkZpSVFqSdxsDgCrSZDfngCojcJczBzrnFHoCAJtCJIq3dWbjRBfP9SbSkLrmZzbCVFp6lHrCBIbDtTtOYLeDdMt7hE&msgID=I6020019772513619968_500&markAsRead=

You received an invitation to connect. LinkedIn will use your email address to make suggestions to our members in features like People You May Know. Unsubscribe here: https://www.linkedin.com/blink?simpleRedirect=ojcJczBzrnFHoCAJtCJIq3dWbjRAqmkCkl5TnQJirzpecn8RpCx5iP1nqjplqQpce5YMtThCgQxmdkUPsTcSgDxhmC9ae3dLmjlxlzdAcRZecAZfqQJGdnEJhRtggQZDl4dchBtGk4hSq515dSxqljxfrRh6gkMRlk51gk5GhT5etCxEuz9mhB51fmhFrSMCcj1vsClArCBJpn9vt7dBtmtvt7dOqmoZqSkCkjoPp4l7q5p6sCR6kk4ZrClHrRhAqmQCrDlIfngCojcJczBzrnFHoCAJtCJIq3dWbjRBfP9SbSkLrmZzbCVFp6lHrCBIbDtTtOYLeDdMt7hE&msgID=I6020019772513619968_500&markAsRead= Learn why we included this at the following link: https://www.linkedin.com/blink?simpleRedirect=e3wTd3RAimlIoSBQsC4Ccj1vsClArCBJpn9vt7dBtmtvt7dOqmoZqSkCr79lpmdFtD9BkT9BrmZQsTlzfm4CojcJczBzrnFHoCAJtCJIq3dWbjRBfP9SbSkLrmZzbCVFp6lHrCBIbDtTtOYLeDdMt7hE&msgID=I6020019772513619968_500&markAsRead=
© 2015, LinkedIn Corporation. 2029 Stierlin Ct. Mountain View, CA 94043, USA


#13

@abrahamavnisan I think http://talk.olab.io can’t join your network. :slight_smile: