Memory usage - Extract vs Select

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

tomfabtastic@hotmail.com - 30 Oct 2008 08:02 GMT

Hello,

Does anyone know what is the most memory efficient way to manipulate
large data sets (200,000 rows and 40 columns) ? I use extract, but
suspect Select may be more efficient.

Also, is there a way to tell how much memory is being used by a
mathematica command ?

Thanks,
Tom

Reply to this Message

Bill Rowe - 31 Oct 2008 09:07 GMT

>Also, is there a way to tell how much memory is being used by a
>mathematica command ?

No. Nor can there be in general. The amount of memory a given
command needs for intermediate steps will often depend on
specifics of the arguments it has to act on. For example,
consider just plotting a function. Since Mathematica samples the
function more where it bends, very non-linear functions will
require more memory than linear functions.

There are ways to determine how much memory is used to store an
expression or how much memory is used by any sequence of
commands after they have completed. ByteCount gives the number
of bytes needed to store a particular expression. MemoryInUse
gives the amount of memory in use at the time this command is
executed. Obviously changes in the value returned by MemoryInUse
are the amount of memory consumed by operations done between
uses of MemoryInUse.

Reply to this Message

Math Forum / Math Software / Mathematica / October 2008

Memory usage - Extract vs Select