Thursday 6 November 2014

Using Memory Mapped files in Java

When two processes map the same file in memory, the memory that one process writes is seen by another process, so memory mapped files can be used as an interprocess communication mechanism. We can say that memory-mapped files offer the same interprocess communication services as shared memory with the addition of filesystem persistence. However, as the operating system has to synchronize the file contents with the memory contents, memory-mapped files are not as fast as shared memory.


/*--------------------------------------------------------------
Class MemoryMapWriter - creates mmap file
---------------------------------------------------------------*/

import java.io.File;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MemoryMapWriter {

public static void main(String[] args) throws Exception {
File f = new File("/tmp/mapped.txt");
f.delete();

FileChannel fc = new RandomAccessFile(f, "rw").getChannel();

int start = 0;
long counter = 1;
long HUNDREDK = 100000;
long startT = System.currentTimeMillis();
long noOfMessage = HUNDREDK * 100 * 10;

long bufferSize = 8 * noOfMessage;
MappedByteBuffer mem = fc.map(FileChannel.MapMode.READ_WRITE, 0,
bufferSize);

for (;;) {
if (!mem.hasRemaining()) {
start += mem.position();
mem = fc.map(FileChannel.MapMode.READ_WRITE, start, bufferSize);
}
mem.putLong(counter);
counter++;
if (counter > noOfMessage)
break;
}
long endT = System.currentTimeMillis();
long tot = endT - startT;
System.out.println(String.format("No Of Message %s , Time(ms) %s ",
noOfMessage, tot));

fc.close();
unmap(fc, mem);
}

private static void unmap(FileChannel fc, MappedByteBuffer bb)
throws Exception {
Class<?> fcClass = fc.getClass();
java.lang.reflect.Method unmapMethod = fcClass.getDeclaredMethod(
"unmap", new Class[] { java.nio.MappedByteBuffer.class });
unmapMethod.setAccessible(true);
unmapMethod.invoke(null, new Object[] { bb });
}

}

/*--------------------------------------------------------------
Class MemoryMapReader - reads mmap file
---------------------------------------------------------------*/
package org.g;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MemoryMapReader {

public static void main(String[] args) throws FileNotFoundException,
IOException, InterruptedException {

FileChannel fc = new RandomAccessFile(new File("/tmp/mapped.txt"), "rw")
.getChannel();

// long bufferSize=8*10000;
MappedByteBuffer mem = fc.map(FileChannel.MapMode.READ_ONLY, 0,
fc.size());
long oldSize = fc.size();

long currentPos = 0;
long xx = currentPos;

long startTime = System.currentTimeMillis();
long lastValue = -1;

int exporterId = 8;

mem.position(exporterId * 8);
long res = mem.getLong() - 1;
if (res != exporterId) {
System.out.println("this should not happen !! ");
}

System.out.println("Resetting pos ...");
// reset position again
mem.position(0);

for (;;) {

while (mem.hasRemaining()) {
lastValue = mem.getLong();
currentPos += 8;
}
if (currentPos < oldSize) {

xx = xx + mem.position();
System.out.println("this shuold never occur !!! ");
// mem = fc.map(FileChannel.MapMode.READ_ONLY,xx, bufferSize);
continue;
} else {
long end = System.currentTimeMillis();
long tot = end - startTime;
System.out.println(String.format(
"Last Value Read %s , Time(ms) %s ", lastValue, tot));
break;
}

}

}

}

More Reading:
--------------------

To unmap the memory mapped byte buffer - http://lotusandjava.blogspot.in/2012/02/how-to-unmap-mappedbytebuffer.html
http://stackoverflow.com/questions/8462200/examples-of-forcing-freeing-of-native-memory-direct-bytebuffer-has-allocated-us


12 comments:

  1. I learned World's Trending Technology from certified experts for free of cost. I got a job in decent Top MNC Company with handsome 14 LPA salary, I have learned the World's Trending Technology from Data science training in btm layout experts who know advanced concepts which can help to solve any type of Real-time issues in the field of Python. Really worth trying Instant Approval Blog Commenting Sites List

    ReplyDelete
  2. Thanks for this post. Really very informative and creative contents. Visit my website to get best Information About
    Best MPSC coaching Centre in Dadar.
    Best MPSC coaching Centre in Dadar
    Top MPSC coaching Center in Dadar

    ReplyDelete
  3. Lakshya IAS Academy is a well-known Institute and the Best IAS Coaching in Mumbai preparing candidates for the Civil Services Examination. It is one of the known IAS coaching centers in Mumbai, which is known for offering high-quality guidance to aspirants.

    ReplyDelete
  4. Lakshya IAS Academy is a well-known Institute and the Best MPSC Coaching in Mumbai preparing candidates for the Civil Services Examination. It is one of the known IAS coaching centers in Mumbai, which is known for offering high-quality guidance to aspirants.

    ReplyDelete