"John " <com.works.math@evans.john> wrote in message
<g6asee$slt$1@fred.mathworks.com>...
> "Oliver Schmitt" <schmitt@med.uni-rostock.de> wrote in
> message <g69oiv$hg7$1@fred.mathworks.com>...
[quoted text clipped - 9 lines]
> Hi Oliver, IMREAD support for JPEG-compressed TIFFs was
> added in R2007b.
Thanks for the hint. I've tested immediately. Unfortunately
it works only for small (approx. 30000x30000 pixels RGB ->
approx. 1GB memory allocation within Matlab) multi-page jpeg
encoded tiff images. If images are larger I get the following:
MATLAB crash file:/home/schmitt/matlab_crash_dump.2711
------------------------------------------------------------------------
Segmentation violation detected at Fri Jul 25
10:09:53 2008
------------------------------------------------------------------------
Configuration:
MATLAB Version: 7.6.0.324 (R2008a)
MATLAB License: 238072
Operating System: Linux 2.6.22.18-0.2-default #1 SMP
2008-06-09 13:53:20 +0200 x86_64
GNU C Library: 2.6.1 stable
Window System: The XFree86 Project, Inc (40300000),
display unix:1000.0
Current Visual: 0x3d (class 4, depth 24)
Processor ID: x86 Family 6 Model 7 Stepping 6,
GenuineIntel
Virtual Machine: Java 1.6.0_01 with Sun Microsystems Inc.
Java HotSpot(TM) 64-Bit Server VM mixed mode
Default Encoding: UTF-8
Fault Count: 1
Register State:
rax = 00000000000000ff rbx = 0000000002fbed60
rcx = 00002aaad719a4a0 rdx = 000000000330c568
rbp = 00000000407f8440 rsi = 0000000000000000
rdi = 00002aab85297260 rsp = 00000000407f8030
r8 = 00002aab33394020 r9 = 000000000000d674
r10 = 0000000000000000 r11 = 00002aaaca5fe1c0
r12 = 00002aab33394020 r13 = 0000000051f03240
r14 = 00000000a3e06480 r15 = 000000000000d674
rip = 00002aaaca5f7611 flg = 0000000000010202
Stack Trace:
[0] wjpg8c.mexa64:mexFunction(3, 0x2b342ad0e6ad,
0x127f407f8400, 0x407f8ce8) + 1653 bytes
[1] libmex.so:mexRunMexFile(16384, 0x407f8cd0,
0x3407f8898, 0x407f8c10) + 75 bytes
[2] libmex.so:Mfh_mex::runMexFileWithSignalProtection(int,
mxArray_tag**, int, mxArray_tag**)(0x407f8a80, 0x407f8680,
0x407f8cd0, 0x01469120) + 137 bytes
[3] libmex.so:Mfh_mex::dispatch_file(int, mxArray_tag**,
int, mxArray_tag**)(0x407f8cd0, 0x3407f8b70, 0x01469120,
0x407f9810) + 262 bytes
[4] libmwm_dispatcher.so:Mfh_file::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0, 3, 0x407f8d00,
0x02556290) + 236 bytes
[5] libmwm_interpreter.so:inDispatchFromStack(int, char
const*, int, int)(0x407f8fec, 0x48900000003, 0x02558710
"wjpg8c", 0) + 1062 bytes
[6] libmwm_interpreter.so:inDispatchCall(char const*, int,
long, int, int*, int*)(0, 0, 0, 0x407f8ff0) + 165 bytes
[7] libmwm_interpreter.so:inInterp(inDebugCheck, int, int,
opcodes, inPcodeNest_tag volatile*, long*)(0x407f9498,
0x2aaace075030, 0x2a00000000, 0x1000003e1) + 4321 bytes
[8] libmwm_interpreter.so:protected_inInterp(inDebugCheck,
int, int, opcodes, inPcodeNest_tag*, long*)(0, 0x02596330,
0x407f9480, 0x2b342cc7e6d0) + 140 bytes
[9] libmwm_interpreter.so:inInterPcodeSJ(inDebugCheck,
int, int, opcodes, inPcodeNest_tag*, long*)(0x02cc1580 ",
0x407f9460, 0x407f94ac, 0x3f00000003) + 265 bytes
[10]
libmwm_interpreter.so:inExecuteMFunctionOrScript(Mfh_mp*,
bool)(0, 0, 0, 0x407f9690) + 680 bytes
[11] libmwm_interpreter.so:inRunMfile(int, mxArray_tag**,
int, mxArray_tag**, Mfh_mp*, inWorkSpace_tag*)(0x407f9a58,
0x300000001, 0x02f6a4f0, 0x407f98d0) + 666 bytes
[12] libmwm_dispatcher.so:Mfh_file::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0xffffffff00000001,
0x2b3435106820 ", 0x407f9990, 0x2b343515c640 ") + 236 bytes
[13] libmwm_dispatcher.so:Mfh_builtin::dispatch_mf(int,
mxArray_tag**, int, mxArray_tag**)(0x407f9a50, 0x4407f98f0,
0x2b343515c640 ", 0x407fa590) + 84 bytes
[14] libmwm_dispatcher.so:Mfh_MATLAB_fn::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0, 4, 0x407f9a01, 0) +
236 bytes
[15] libmwm_interpreter.so:inDispatchFromStack(int, char
const*, int, int)(0x407f9d6c, 0x125fffffffc, 0x0241b2dc
"feval", 0) + 1062 bytes
[16] libmwm_interpreter.so:inDispatchCall(char const*,
int, long, int, int*, int*)(0, 0x02675ec0, 50, 0x407f9d70) +
165 bytes
[17] libmwm_interpreter.so:inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag volatile*, long*)(0x407fa218,
0x2aaace0758d0, 0x16b00000000, 0x10000024a) + 4321 bytes
[18]
libmwm_interpreter.so:protected_inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag*, long*)(0x407fa030,
0x2b342be88663, 0x02cabe90, 0x008f9ad0) + 140 bytes
[19] libmwm_interpreter.so:inInterPcodeSJ(inDebugCheck,
int, int, opcodes, inPcodeNest_tag*, long*)(0x02ce9880 ",
0x407fa1e0, 0x407fa22c, 0x2b342b1a8a2b) + 265 bytes
[20]
libmwm_interpreter.so:inExecuteMFunctionOrScript(Mfh_mp*,
bool)(0, 0, 0x407fa440, 0x407fa410) + 680 bytes
[21] libmwm_interpreter.so:inRunMfile(int, mxArray_tag**,
int, mxArray_tag**, Mfh_mp*, inWorkSpace_tag*)(0x407faa00,
0x2407faba0, 0x02deafd0, 0x407fc1a0) + 666 bytes
[22] libmwm_dispatcher.so:Mfh_file::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0x407fa650,
0xffffffff00000000, 0x407fa630, 0) + 236 bytes
[23]
libmwm_interpreter.so:ResolverFunctionDesc::CallFunction(int,
mxArray_tag**, int, mxArray_tag**)(2, 0x7300000000000002,
0x407fb060, 0x407fabb0 ") + 786 bytes
[24] libmwm_interpreter.so:Resolver::CallMFunction(int,
int, _m_operand*, m_operand_storage*, int, _m_operand*,
m_operand_storage*, int*)(0x02eddd70, 0, 0x407fb6b0,
0x407fb0e1) + 1672 bytes
[25]
libmwm_interpreter.so:inResolveMFunctionCall(_m_function_desc*,
int, int, _m_operand*, m_operand_storage*, int, _m_operand*,
m_operand_storage*, int*, inMarshalType*, int,
mpsTypeSequenceNlhs const*, mxArray_tag*
(*)(int))(0x02eddd70, 0, 0x407fb6b0, 0x407fb848) + 517 bytes
[26]
libmwm_interpreter.so:accelImpl::MFunctionCall(_accelOp**)(3,
0, 0x9407fb6f0, 0x2aaacda6eed0) + 242 bytes
[27] libmwm_interpreter.so:accelImpl::Exec()(0x407fb848,
0x407fb840, 0x02de9690, 0x02c8b070) + 238 bytes
[28] libmwm_interpreter.so:accelCode::Call(inMarshalType*,
int*) const(1, 0x407fb84f, 33, 0x0064b800 ") + 91 bytes
[29]
libmwm_interpreter.so:inJit::ExecuteHotSegment(_inJitAccelInfo*,
opcodes*, int*, long*)(0x407fbbb0, 0, 0x2b342cc26780,
0x407fb980) + 1664 bytes
[30] libmwm_interpreter.so:inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag volatile*, long*)(0x407fbe28,
0x2aaace076b30, 0x200000000, 0x100000000) + 6253 bytes
[31]
libmwm_interpreter.so:protected_inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag*, long*)(0x407fbc10,
0x2b342bea4481, 0, 23) + 140 bytes
[32] libmwm_interpreter.so:inInterPcodeSJ(inDebugCheck,
int, int, opcodes, inPcodeNest_tag*, long*)(0, 0x407fbdf0,
0x407fbe3c, 0x3f00000000) + 265 bytes
[33]
libmwm_interpreter.so:inExecuteMFunctionOrScript(Mfh_mp*,
bool)(0, 1, 0x01344320, 0x407fc020) + 680 bytes
[34] libmwm_interpreter.so:inRunMfile(int, mxArray_tag**,
int, mxArray_tag**, Mfh_mp*, inWorkSpace_tag*)(0x407fc320,
0x407fc1c0, 0x02c1bed0, 0x407fce60) + 666 bytes
[35] libmwm_dispatcher.so:Mfh_file::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0x407fc250, 0,
0x407fc200, 0x2b342ae88b80) + 236 bytes
[36] libmwm_interpreter.so:inDispatchFromStack(int, char
const*, int, int)(0x407fc63c, 0x47e00000000, 0x02e5889c
"extract_ImageLevels", 0x100000000) + 1062 bytes
[37] libmwm_interpreter.so:inDispatchCall(char const*,
int, long, int, int*, int*)(0, 0x03083280, 0x2aaa00000004,
0x407fc640) + 165 bytes
[38] libmwm_interpreter.so:inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag volatile*, long*)(0x407fcae8,
0x2aaace07bc60, 0x100000000, 0x100000000) + 4321 bytes
[39]
libmwm_interpreter.so:protected_inInterp(inDebugCheck, int,
int, opcodes, inPcodeNest_tag*, long*)(0x02c434a0,
0x008f9ad0, 0x02c434a0, 0x13040b227bae36) + 140 bytes
[40] libmwm_interpreter.so:inInterPcodeSJ(inDebugCheck,
int, int, opcodes, inPcodeNest_tag*, long*)(0, 0x407fcab0,
0x407fcafc, 0) + 265 bytes
[41]
libmwm_interpreter.so:inExecuteMFunctionOrScript(Mfh_mp*,
bool)(0, 1, 0x2aaac424ee30, 0x407fcce0) + 680 bytes
[42] libmwm_interpreter.so:inRunMfile(int, mxArray_tag**,
int, mxArray_tag**, Mfh_mp*, inWorkSpace_tag*)(0,
0x2d78d960, 0x02c434a0, 0x407fde00) + 666 bytes
[43] libmwm_dispatcher.so:Mfh_file::dispatch_fh(int,
mxArray_tag**, int, mxArray_tag**)(0x407fcf00, 0,
0x407fdbc0, 0x407fda40) + 236 bytes
[44]
libmwm_interpreter.so:inEvalPcodeHeaderToWord(_memory_context*,
int, mxArray_tag**, _pcodeheader*, Mfh_mp*, unsigned
int)(0x407fcf20, 0x407fd0c0, 0x407fdc10, 0) + 187 bytes
[45]
libmwm_interpreter.so:in_local_call_script_function(_memory_context*,
_pcodeheader*, int, mxArray_tag**, unsigned int,
bool)(0x407fd920, 0x407fdad0, 0x407fd9c0, 0x02a0db70) + 103
bytes
[46]
libmwm_interpreter.so:inEvalStringWithIsVarFcn(_memory_context*,
char const*, EvalType, int, mxArray_tag**, inDebugCheck,
_pcodeheader*, int*, bool (*)(void*, char const*), void*,
bool, bool)(0, 0, 0x2b342bc7ff60, 0) + 1529 bytes
[47] libmwm_interpreter.so:inEvalCmdWithLocalReturn(char
const*, int*, bool, bool, bool (*)(void*, char
const*))(0x407fdc80, 0x0258fc90 "extract_ImageLevels\n",
0x407fddc0, 0) + 149 bytes
[48] libmwbridge.so:evalCommandWithLongjmpSafety(char
const*)(0, 0, 0, 0) + 94 bytes
[49] libmwbridge.so:mnParser(0x00580860, 0x005809f0,
0x408000d0, 0x7fff80011e00) + 291 bytes
[50]
libmwmcr.so:mcrInstance::mnParser()(0x636f6c2f7273752f,
0x616c74616d2f6c61, 0x2f61383030325262, 0x786e6c672f6e6962)
+ 82 bytes
This error was detected while a MEX-file was running. If the
MEX-file
is not an official MathWorks function, please examine its
source code
for errors. Please consult the External Interfaces Guide for
information
on debugging MEX-files.
If it is an official MathWorks function, please
follow these steps to report this problem to The MathWorks so we
have the best chance of correcting it:
The next time MATLAB is launched under typical usage, a
dialog box will
open to help you send the error log to The MathWorks.
Alternatively, you
can send an e-mail to segv@mathworks.com with the following
file attached:
/home/schmitt/matlab_crash_dump.2711
If the problem is reproducible, please submit a Service
Request via:
http://www.mathworks.com/support/contact_us/ts/help_request_1.html
A technical support engineer might contact you with further
information.
Thank you for your help. MATLAB may attempt to recover, but
even if recovery appears successful,
we recommend that you save your workspace and restart MATLAB
as soon as possible.
Rune Allnor - 25 Jul 2008 11:00 GMT
> "John " <com.works.m...@evans.john> wrote in message
>
[quoted text clipped - 16 lines]
> approx. 1GB memory allocation within Matlab) multi-page jpeg
> encoded tiff images. If images are larger I get the following:
[...]
If an image of 1GB memory footprint is considered 'small' I
suspect you have few options but to buy a new computer with
64 bit OS and plenty of RAM.
The thing is that you have hit a limit somewhere. If you use
a 32Bit platform, the max RAM that can be handled is 2GB per
CPU. Including disk swap space. Any larger than that, and you
will have to divide whatever data in smaller blocks.
If you work with a single-core computer the CPU has at most
2GB RAM to handle everything that goes on on the computer,
like OS, graphics, matlab anything else. With multi-CPU
computers you have more space alltogether, but any one
core can only handle at most 2GB RAM.
Rune
Oliver Schmitt - 25 Jul 2008 11:30 GMT
Rune Allnor <allnor@tele.ntnu.no> wrote in message
<d73b123e-b6ae-4589-b5a7-8b092ee14898@q28g2000prh.googlegroups.com>...
> > "John " <com.works.m...@evans.john> wrote in message
> >
[quoted text clipped - 34 lines]
>
> Rune
Thank you for help. I'm using a 64Bit platform with 64GB
RAM. I found that imread is not the problem. Actually it can
read images up to 100000x50000 pixel RGB (tested so far) and
matlab allocates 16 GB memory.
The crash documented above results from the trial to write
the decoded image data to a single jpeg image. The imwrite
function with 'jpeg' option was responsible. Using png we
are able to store up to 2^32 byte. So this is the new
problem the images are larger than 2^32 and jpeg compression
is not possible. Has anyone an idea how to write an image
that is larger than 2^32 byte?
John Evans - 28 Jul 2008 20:57 GMT
> Thank you for help. I'm using a 64Bit platform with 64GB
> RAM. I found that imread is not the problem. Actually it can
[quoted text clipped - 7 lines]
> is not possible. Has anyone an idea how to write an image
> that is larger than 2^32 byte?
Oliver, if the file size exceeds 2GB, look into saving your image in an HDF5
file. You could use HDF5WRITE, but the low-level HDF5 interface would give
you more control over things like chunk size, compression, etc.
Oliver Schmitt - 28 Jul 2008 22:09 GMT
John Evans <John.Evans@mathworks.com> wrote in message
<g6l8b5$nm5$1@fred.mathworks.com>...
> > Thank you for help. I'm using a 64Bit platform with 64GB
> > RAM. I found that imread is not the problem. Actually it can
[quoted text clipped - 11 lines]
> file. You could use HDF5WRITE, but the low-level HDF5 interface would give
> you more control over things like chunk size, compression, etc.
I found a solution:
unix(sprintf('tiffcp -r 16 -c jpeg *.tif 9layer.tif'))
Here 9 *.tif images are jpeg compressed and written to a
multi-page tif. This works also with very large images.
Thanks for help.
Peter Boettcher - 25 Jul 2008 14:16 GMT
> The thing is that you have hit a limit somewhere. If you use
> a 32Bit platform, the max RAM that can be handled is 2GB per
[quoted text clipped - 6 lines]
> computers you have more space alltogether, but any one
> core can only handle at most 2GB RAM.
Memory limitations have nothing to do with number of CPUs/cores.
"32-bit computer" specifies only the amount virtual memory space
available to a process. Windows, by default, uses one bit to
distinguish between kernel memory and user memory in the same process,
leaving the user with 31 bits of address, or 2 GB. That space is shared
by all the stuff in that process, like MATLAB program code, the Java VM,
data, etc. And since Windows loads some of this stuff (like the program
code) in the MIDDLE of the available user space, the largest contiguous
block (and thus the largest MATLAB variable) available is usually lower
than 2GB. The often-discussed /3GB switch to Windows shifts the
user/kernel division to 3/1 instead of 2/2.
Even on a 32-bit CPU/OS, this process limitation is not the same as a
physical memory constraint. If you have 4GB of memory, you are still
limited by the 2 (or 3) GB address, but you can run other programs (with
their own 2 or 3 GB address space) in what remains of physical memory.
Some hardware can even play tricks to support > 4GB of physical memory,
while still being limited to 4GB per process.
Full 64-bit platforms eliminate all this, as virtual addresses are 64
bits, which not only allows user programs to reach all the physical
memory on the system, but allows plenty of space to do things like map
terabyte files into memory.
-Peter