Monday 20 April 2015

memory consumption and processing times

To measure a time taken by a process the following function can be useful

system.time() e.g.   
system.time(sum(seq(1,10000000,by=1)))     
   user  system elapsed
   0.44    0.08    0.53  


To display the output of the process as well, use print() inside

system.time(print(sum(seq(1,10000000,by=1))))  
[1] 5e+13
   user  system elapsed
   0.41    0.02    0.42  
   


To check the size of the object, use object_size() from 'pryr' package

library(pryr)   
object_size(iris)   
7.03 kB

To check how much memory has been used by the session,

mem_used()   
21 MB

To check how much memory change a particular process makes,

mem_used()   
21 MB
mem_change(1+100)  
10.6 kB   
mem_used()   
21.1 MB


Garbage collection gc() runs automatically in R to release memory when an object is removed. You can still run this command manually.

mem_used()   
21.1 MB 
dat<-iris[rep(1:nrow(iris),1000),]
mem_used()  
37.3 MB 
rm(dat)   
mem_used() 
24 MB 
gc()   
         used (Mb) gc trigger (Mb) max used  (Mb)
Ncells 343524 18.4     741108 39.6   707448  37.8
Vcells 591474  4.6    2028694 15.5 21067777 160.8 
mem_used()   
24 MB


For Windows, to check memory in use and change limits of memory allocation
 
library(utils)  

 
To check size of memory currently in use
 
memory.size(max=FALSE)  

 
To check maximum amount of memory obtained from OS
 
memory.size(max=TRUE)  

e.g.
#memory immediately after opening a new R session  
memory.size(max=TRUE)    
[1]  16.56 

memory.size(max=FALSE)   
 [1]  12.58 

dat<-matrix(1:2700, 30, 90)  #create a matrix 30 x 90  

memory.size(max=TRUE)  #maximum memory obtained remains same  
[1]  16.56  

memory.size(max=FALSE)  #memory currently in use increased after the matrix was created  
 [1]  12.61  
 

To change memory allocation limit to specific amount (in Mb)
 
memory.limit(size=8000)  
   

To check current memory allocation limit  
 
memory.limit(size=NA)  

e.g.

#at the opening of a new R session (32 bit)
memory.limit(size=NA)  
[1]  3583   
memory.limit(size=3600)  #increased size to 3600  
[1]  3600  
memory.limit(size=NA)   
[1]  3600   
memory.limit(size=3800)  #increased size to 3800  
[1]  3800   
memory.limit(size=NA)   
[1]  3800   

N.B. the memory allocation limit cannot be set to a lower limit (i.e. you can only increase it)




No comments:

Post a Comment