Issues with large files and containers, C++

Development Code
1

Hey guys,


I've been plugging away at this and keep running into compiler errors or the program simply not functioning as intended.


I'm generating chains of numbers from file1 and comparing and checking them to file2. If the generated chain is within certain lengths then it's output to a file containing the newly generated strings.


For the record, file1 is 8k integers and file2 is 27mil integers.
I need to do this for 3 different mathematical formulas while still using the same table of numbers.


The integers are generally stored in a file in this format and are sorted numerically from low to high:

1 2 3 4 5 6 7 8

9 10 11 12 13 14 15 16

...

...

Although sometimes there are extra spaces because running Replace with notepadd++ across 27mil integers causes freezes.

I'm mostly using vectors here but not being able to easily search them is a pain and doing a binary search on a 27mil element vector up to 10 times each of the 8k elements in the original vector is... computer melting.


So what kind of container should I use?
I only have ints going into the dataFile containers so this container doesn't need to be templated out with multiple members it just needs to be quickly accessed by index and by searching.


I've been wailing away at this for 3 days now and it either gives me compiler errors because xContainer can't compare int/iterator/whatever or it's not indexing and pulling the files correctly or something along those lines.


I could use some help.

What i'm trying: find it here (http://forums.devshed.com/c-programming-42/trouble-comparing-elements-of-large-files-955445.html#post2916140)

 

2

Yeah, you're going to have to reformat that. Use PasteBin or even GitHub.

3

Use pastecode or codebin or any other service? Impossible to read. Make it easy to help you.

4

10 edits later its fixed.

 

update: updated the code on linked page, realized it wasn't my current version

5

Alright, so i think have the search function to check against the vector (not using find() which was just for some readability) what i really need is a way to read the text from the file and pull the integers out assigning each one to the vector.

So:

  1. Open File
  2. Stream Data
  3. do{
  4. parse integers from string
  5. vector.push_back(integers);
  6. }while (!File.eof());

I've searched for hours for a method to do this but it either doesn't compile or it vomits out random numbers from the file into the array.

got it all working.