Task #9691 (closed)
Opened 7 years ago
Closed 7 years ago
RFE: Use memory mapped files to reduce memory footprint
| Reported by: | rleigh | Owned by: | mlinkert |
|---|---|---|---|
| Priority: | minor | Milestone: | Unscheduled |
| Component: | Bio-Formats | Version: | n.a. |
| Keywords: | n.a. | Cc: | |
| Resources: | n.a. | Referenced By: | n.a. |
| References: | n.a. | Remaining Time: | n.a. |
| Sprint: | n.a. |
Description
Following discussion with josh this morning:
If readers could be modified to directly access file data through memory mapped I/O, this will reduce memory usage in many cases since the mapped image files pages will not contribute to heap usage, and can be dropped from the page cache as needed, so will not result in heavy swapping if pressure for memory is high. This should make readers both faster and use less memory.
While this won't greatly benefit readers which need to decompress the entire file in memory, it does save the need to have any of the compressed data on the heap.
Maybe we could adapt the most memory-hungry readers initially, and perhaps the generic readers used by many readers (such as the tiff readers).
Roger
Change History (3)
comment:1 Changed 7 years ago by jmoore
comment:2 Changed 7 years ago by mlinkert
- Summary changed from Bug: Use memory mapped files to reduce memory footprint to RFE: Use memory mapped files to reduce memory footprint
comment:3 Changed 7 years ago by mlinkert
- Resolution set to fixed
- Status changed from new to closed
Referencing ticket #8731 has changed sprint.