An enhanced file storage implementation in MongoDB that extends the GridFS functionality of the Mongo-Java-Driver
This project came out of a need in my company to extend the current GridFS functionality to include compression of file data. After spending a day or so trying to work with the existing implementation, I determined that the existing implementation was not going to be extendible due to the current use of inner classes and heavy cross-class collusion. So I decided to re-write the existing implementation in order to extend it and publish those enhancements for anyone else who will like to make use of it.
Also the "coming soon" Mongo-Java-Driver 3.0.x which is an almost complete re-write by the MonogDB folks has a new and much cleaner API and Object Model means that I need a way to bridge the gap seamlessly at my current company because we are making heavy use of the library using the 2.10.x thru 2.12.x drivers. I need to be able to migrate these systems to the new 3.0.x driver when it is released.
Goto the downloads section to see the latest version and how to get it. Check out my task list under the TODO to check the progress.
| Feature | GridFS ( 2.11.4 ) | mongoFS ( 0.8.x ) |
|---|---|---|
| Built into the MongoDB driver | ||
| Store files in MongoDB | ||
| MD5 data checksums | ||
| Available in JVM based languages | ||
| Available in other languages | Not Yet | |
| Optimized for usePowerOf2Sizes storage | starting in 2.6 | supported in 2.4+ |
| Background deletes ( asynchronous ) | ||
| Temporary file storage ( expiration ) | ||
| Provides URL syntax for easy file reference | ||
| Space saving data compression ( gzip ) | ||
| Storing files with customer defined encryption | ||
| 3.0.x driver compatible objects with 2.12.x driver | 0.8.x | |
| ZIP archive expansion and manifests | 0.9.x | |
| Java 3.0 driver compatible | Coming ( 1.0.x ) | |
| Non-blocking IO and reactive | Coming ( 1.1.x ) |
mongoFS was built to be compatible with an existing GridFS collections. There is even an implementation of GridFS inside of mongoFS that has been refactored to use some mongoFS classes underneath.
Simply point MongoFileStore to the existing GridFS collections and gain the new compression, excryption and MongoFile URL functionality. A GridFS compatiable configuration is provided to disable the encryption and compression support.
NOTE: Once compression or encryption is enabled, this is a one-way process. Once mongoFS starts writing compressed and/or encrypted file chunks, the data read from GridFS directly will be a Gzip compressed and/or an encrypted stream that you will have to handle manually to get the raw data back. Also, mongoFS adds fields to the documents that help it know what to do with the file and chunk data in the DB collections.
This migration will require some coding changes need to ensure compatiblity with the uncoming MongoDB 3.0.x Java driver. The code changes allow for the upcoming change from DBObject to the new Document base object found in the 3.0.x driver. Most of the changes fro the 3.0.x driver are hidden from the user but some changes to my API were needed to support the new class structure in the 3.0.x driver.
mongoFS is now available in the Maven Central Repository
Older versions can be found in my maven based repo is hosted on GitHub,
the resolver URL for that repo is
https://github.com/dbuschman7/mvn-repo/raw/master
To see all versions available in Maven Central, go to MvnRepository.com
<dependency>
<groupId>me.lightspeed7</groupId>
<artifactId>mongoFS</artifactId>
<version>0.8.1</version>
</dependency>
You can get the source code directly from GitHub.
Visit Github Page