Distributed file system with local disk cache
Solution 1:
Check out OpenAFS it has local disk cache, see: http://docs.openafs.org/Reference/5/afs_cache.html
Solution 2:
Another contender is XtreemFS: the feature set includes
In addition to full replicas that contain a complete copy, XtreemFS also supports partial replicas. These replicas are filled on demand when a client accesses data.
Solution 3:
As per comments elsewhere - it would be possible to use local disk storage for caching gluster I/O albeit at the cost of VFS cache, AFS seems to be apropriate. But the big omissions from your question is whether you are trying to achieve fault-tolerance or performance, and whether the replicated storage should support transactions or frequent writes.
Other options include
- using a replicating nosql database
- bcache (which will provide performance improvements but not resillience improvements and poses problems with frequent writes / cache consistency)
- NAS/SAN