Memory Systems Research: An Industry Perspective

After several months of informal meetings and ad-hoc contact, we finally visited memory manufacturer Micron in Boise, ID earlier this month. Our large group of 7 students and 2 faculty members made for a fun road trip through stretches of torrential rain and awesome homemade pie at Connor’s Cafe in Burley (I would definitely recommend you stop by if you’re ever on I-84 in Idaho!). In a 3+ hour presentation and discussion session, we discussed all of the memory research happening here at our group in Utah, including both published work and current work-in-progress. We wanted feedback on what the memory industry’s constraints were, and the feasibility of implementing some of our ideas, and here are some of the things we heard, in no particular order:

1. Density, density, density: That density and cost/bit are everything in the memory industry is well known in memory research circles, but we really had no idea how much! We were told that on commodity parts, a 0.01% area penalty “might be considered”, 0.1% would require “substantial” performance gains, and anything over 0.5% was simply not happening, no matter what (these numbers are meant to be illustrative, of course, and not precise guidelines). For more specialized parts for niche markets, on the other hand, slightly larger area penalties could be considered as part of the overall value package. Take away: Focus on the high-end specialized segment, say the next big supercomputer, where cost is a smaller concern. The commodity segment has been squeezed to death, and you’re probably not going to get anywhere trying to change anything.

2. Be afraid of the memory chip: Very afraid! This is related to the previous point. The actual array layout has been super-ultra-optimized by people who dream about this stuff, and even the slightest change you propose is likely going to mess with this beyond repair. Stay away! Also, lots of smart people care very deeply about this narrow field, and have been working on this particular aspect for a long time. Anything you think of, they’ve probably considered before (but not published!). They would be “astounded” to hear something really novel.

3. Work at the system-level: There are infinite possibilities in terms of workload-based studies, altering data placement, data migration, row-buffer management, activity throttling, etc. These are less likely to provide dramatic improvements, but are low-effort, and are more likely to have an impact in terms of actually being implemented, since they are less invasive. It is unlikely that the industry has studied all the possibilities exhaustively, and you’re much more likely to come up with something novel.

4. Build it and they will come. NOT. If you really feel there is a case to be made to radically change DRAM architecture, approach the guys that will actually buy and use the parts - the system manufacturers. If they care enough, they can ask for it, and if they’re willing to pay for it, Micron will build it. It cannot be driven from the other end, no matter what, the margins are simply too small, and the whole system is setup to incentivize maximum capacity at least cost.

5. The Universal Memory Myth: Micron has been researching PCM for over a decade now, and are only now kinda sorta ready-ish to release a product into the market. Their excitement about the maturity of the technology, and it’s ability to summarily replace all memory in the system, is far far lower than that of the academic community. It is nowhere close to reaching the capacity and cost point of DRAM, which is, therefore, not likely to die any time soon.

Dude, where are my PCM parts going?

All of us have heard of the not-so-new buzz surrounding the slew of new-er non-volatile memories (NVMs) that have either hit the market, or are expected to do so soon. The noteworthy among these are Phase-change memory (PCM) and an advancement of the Magnetoresistive Random Access Memory (MRAM) called Spin-Torque Transfer RAM (STT-RAM). PCM parts have started to trickle out as commercial products, while the viability of STT-RAM parts is still being demonstrated. Although, from the excitement about STT-RAM at the Non-Volatile Memories workshop this year, I am sure that we will be seeing more and more of it in the coming years.

Recently, the architecture community has been abuzz with proposals on applications of  PCM, at different levels in the memory and storage hierarchy. But, the buzz notwithstanding, let's take a quick look at the PCM products that are presently available. Numonyx has 128 Mb serial and parallel parts on the market. While there has been talk of Samsung shipping a 512 Mb part, the part information is unavailable, as far as I know. Although, it does seem like Samsung is "shipping a little bit" of PCM parts. Where, to whom, and what for is still not very clear. But there have been reports of PCM parts actually being used in consumer electronics. From the current state of things, it seems like the consumer electronics industry (cameras, cell phones and the likes) is intent on using PCM as a plug-and-play replacement for Flash. Assuming that manufacturers come out with high density, low latency PCM parts, how are they best used in desktops and server-side machines?

Industry analysts believe that PCM is a good contender to replace Flash in the short term. Taking that thought a little further, IBM believes that PCM will have a big role to play in the server-side storage hierarchy, but again, as a replacement for Flash SSDs. I tend to agree with them, at least in the short run. Many research papers advocate that PCM will eventually serve as a DRAM replacement.  Thus, there is little consensus on how and where PCM parts may fit in the memory/storage hierarchy.

From what I have learned so far, there are four main issues that will have to be handled before PCM is ready to be a main memory replacement - (i) endurance has to reach somewhere around that of DRAM, (ii) write energy and latency has to be reduced substantially, (iii) higher levels of the hierarchy have to be more effective at filtering out writes to the PCM level, and (iv) the recently discovered problem of resistance drift with multi-level cells has to be addressed. More recently, there have been concerns about scaling of PCM devices because of issues with increasing current densities and electromigration issues (Part 1, Part 2, Part 3). This, of course is assuming that all issues associated with wear-leveling have been worked out.

Apart from the issues with the devices themselves, has there been a general reluctance to shift to PCM? To quote Samsung's top honcho on the semiconductor side of the business "systems guys have been very reluctant to adopt the technology in mass quantities". The quote conveys very little discernible information, but, as a first guess, it might mean that the system software stack has to be extensively reworked to make PCM a reality.

There are many views on the future of PCM.  There appears to be a slight disconnect between industrial trends and academic thrusts.  The former seems to focus on using PCM as a permanent storage device, while the latter is attempting to move PCM higher up the hierarchy.  A case can be made that more academic effort must be invested in developing PCM as a viable storage class memory.