Using Redis supported by NVRAM in HPC applications

Artur Malinowski

doi:10.7494/csci.2017.18.3.2077

Authors

Artur Malinowski Gdansk University of Technology, Faculty of Electronics, Telecommunications and Informatics, Gdansk, Poland

DOI:

https://doi.org/10.7494/csci.2017.18.3.2077

Keywords:

high performance computing, storage systems, NoSQL, NVRAM

Abstract

Nowadays, the efficiency of a storage systems is a bottleneck in many moern HPC clusters. High performance in traditional approach – processing using files – is often difficult to obtain because of model complexity and its read/write patterns. Alternative approach is applying a key-value database, which usually has low latency and scales well. On the other hand, many key-value stores suffer from limitation of memory capacity and vulnerability to serious faiures, which is caused by processing in RAM. Moreover, some research suggests, that scientific data models are not applicable to storage structures of key-value databases. In this paper, the author proposes resolving mentioned issues by replacing RAM with NVRAM. Practical example is based on Redis NoSQL. The article contains also a three domain specific APIs, that show the idea bhind transformation from HPC data model to Redis structures, as well as two micro-benchmarks results.

Downloads

Download data is not yet available.

References

Butler D.M.: Scientific Computing Doesn’t Need noSQL. In: Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC ’12, pp. 1301–1302. IEEE Computer Society, Washington, DC, USA, 2012. ISBN 978-0-7695-4956-9. URL http://dx.doi.org/10.1109/SC. Companion.2012.158.

Foong A., Hady F.: Storage As Fast As Rest of the System. In: 2016 IEEE 8th International Memory Workshop (IMW), pp. 1–4. 2016. URL http://dx.doi. org/10.1109/IMW.2016.7495289.

Forum M.P.I.: MPI: A Message-Passing Interface Standard Version 3.1, 2015. http://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf.

Gao S., Xu J., Hrder T., He B., Choi B., Hu H.: PCMLogging: Optimizing Transaction Logging and Recovery Performance with PCM. In: IEEE Transactions on Knowledge and Data Engineering, vol. 27(12), pp. 3332–3346, 2015. ISSN 1041-4347. URL http://dx.doi.org/10.1109/TKDE.2015.2453154.

Han J., E H., Le G., Du J.: Survey on NoSQL database. In: Pervasive Computing and Applications (ICPCA), 2011 6th International Conference on, pp. 363–366. 2011. URL http://dx.doi.org/10.1109/ICPCA.2011.6106531.

Hanlon M.R., Dooley R., Mock S., Dahan M., Nuthulapati P., Hurley P.: A Case Study for NoSQL Applications and Performance Benefits: CouchDB vs. Postgres. In: Proceedings of the 2011 TeraGrid Conference: Extreme Digital Discovery. ACM, New York, NY, USA, 2011. ISBN 978-1-4503-0888-5.

Intel Corporation: Introducing Breakthrough Memory Technology, 2015. http: //www.intel.com/content/www/us/en/architecture-and-technology/non-volatile-memory.html.

Kryder M.H., Kim C.S.: After Hard Drives. What Comes Next? In: IEEE Transactions on Magnetics, vol. 45(10), pp. 3406–3413, 2009. ISSN 0018-9464. URL http://dx.doi.org/10.1109/TMAG.2009.2024163.

Li T., Verma R., Duan X., Jin H., Raicu I.: Exploring Distributed Hash Tables in HighEnd Computing. In: SIGMETRICS Perform. Eval. Rev., vol. 39(3), pp. 128–130, 2011. ISSN 0163-5999. URL http://dx.doi.org/10.1145/2160803. 2160880.

Mateescu G., Gentzsch W., Ribbens C.J.: Hybrid ComputingWhere {HPC} meets grid and Cloud Computing. In: Future Generation Computer Systems, vol. 27(5), pp. 440 – 453, 2011. ISSN 0167-739X. URL http://dx.doi.org/http://dx.doi.org/10.1016/j.future.2010.11.003.

Mohan C.: History Repeats Itself: Sensible and NonsenSQL Aspects of the NoSQL Hoopla. In: Proceedings of the 16th International Conference on Extending Database Technology, EDBT ’13, pp. 11–16. ACM, New York, NY, USA, 2013. ISBN 978-1-4503-1597-5. URL http://dx.doi.org/10.1145/2452376. 2452378.

Monnerat L., Amorim C.L.: An Effective Single-hop Distributed Hash Table with High Lookup Performance and Low Traffic Overhead. In: Concurr. Comput. : Pract. Exper., vol. 27(7), pp. 1767–1788, 2015. ISSN 1532-0626. URL http: //dx.doi.org/10.1002/cpe.3342.

Oral S., Dillow D.A., Fuller D., Hill J., Leverman D., Vazhkudai S.S., Wang F., Kim Y., Rogers J., Simmons J., Miller R.: OLCFs 1 TB/s, Next-Generation Lustre File System. In: Proceedings of Cray User Group Conference (CUG 2013). 2013.

Patterson D.: Past and Future of Hardware and Architecture. In: SOSP History Day 2015, SOSP ’15, pp. 9:1–9:63. ACM, New York, NY, USA, 2015. ISBN 978-1-4503-4017-5. URL http://dx.doi.org/10.1145/2830903.2830910.

Qiu M., Ming Z., Li J., Gai K., Zong Z.: Phase-Change Memory Optimization for Green Cloud with Genetic Algorithm. In: IEEE Transactions on Computers, vol. 64(12), pp. 3528–3540, 2015. ISSN 0018-9340. URL http://dx.doi.org/10.1109/TC.2015.2409857.

Strohmaier E., Dongarra J., Simon H., Meuer M.: TOP 10 Sites for June 2016. https://www.top500.org/lists/2016/06/. Accessed: 2016-07-10.

Zhao B.Y., Huang L., Stribling J., Rhea S.C., Joseph A.D., Kubiatowicz J.D.: Tapestry: A Resilient Global-scale Overlay for Service Deployment. In: IEEE Journal on Selected Areas in Communications, vol. 22(1), pp. 41–53, 2006. ISSN 0733-8716. URL http://dx.doi.org/10.1109/JSAC.2003.818784.

Zhao B.Y., Kubiatowicz J.D., Joseph A.D.: Tapestry: An Infrastructure for Fault-tolerant Wide-area Location and. Tech. rep., Berkeley, CA, USA, 2001.