Saturday, 31 August 2013

Read File in chunks and then read line by line using LineNumberReader. Repeat this activity

Read File in chunks and then read line by line using LineNumberReader.
Repeat this activity

I have a file containing some 6.5 lakh lines. Now I wish to read every
line of this file using LineNumberReader.
However I am encountering an outofMemoryError on adding these many number
of lines to another 3rd party library..
What I intend to do is, read 200000 lines of a file at a time and add
these lines to 3rd party library.
I am using LineNumberReader but I think the entire file is being read
although I have provided condition that when line count reaches 200000
break the loop and add these to 3rd party library..
A code snippet for the same:
LineNumberReader lnr=new LineNumberReader(new FileReader(file));
String line=null;
int i=0;
while(flags)
{
while( null != (line = lnr.readLine()) ){
i++;
3rdPartyLibrary.add(line.trim());
if(i==200000)
{
System.out.println("Breaking");
lnr.mark(i);
break;
}
if(i==400000)
{
System.out.println("" );
lnr.mark(i);
break;
}
if(i==600000)
{
System.out.println("BREAKING " );
lnr.mark(i);
break;
}
}
if(line==null)
{
System.out.println(" FLAG");
flags=false;
}
lnr.reset();
}
What I am intending to do here is read file from 0-200000 in first
iteration. Then read each individual line and add to 3rd party lib.. Once
this is done, read another 200000 lines from (200001-400000) and then
repeat the same activity.
Need help..Can someone please guide..

No comments:

Post a Comment