楼主: harlon1976
4005 21

[学科前沿] 大量循环时程序怎么就报错呢?少量循环是可以的。 [推广有奖]

11
数据分析师3K 发表于 2012-4-20 16:23:36
试了一下,还是放弃!  ╮(╯▽╰)╭

12
数据分析师3K 发表于 2012-4-20 16:30:32
SAS/IML的编程语言距离MATLAB、GAUSS之类的相差太远了

13
harlon1976 发表于 2012-4-20 16:57:29
现在的问题不是程序正确与否的问题,能够运行就是没有问题,现在的问题是如何优化使之能够运行大量次数循环的问题了,希望其他高手给予帮助。

14
水水尤寒 发表于 2012-4-20 18:46:05
内存不够,我用R算循环时也有这种情况。只能小批量的循环,分批把结果输出就行了

15
harlon1976 发表于 2012-4-20 19:20:21
这个不是解决问题 的办法,应该有更好的办法

16
bobguy 发表于 2012-4-21 06:30:55
Take this first if it helps.

*****************************
Memory and Workspace
You do not need to be concerned about the details of memory usage in IML, because memory allocation is done automatically. However, if you are interested, the following sections explain how it works.

There are two logical areas of memory, symbol space and workspace. Symbol space contains symbol table information and compiled statements. Workspace contains matrix data values. Workspace itself is divided into one or more extents.

At the start of a session, the symbol space and the first extent of workspace are allocated automatically. More workspace is allocated as the need to store data values grows. The SYMSIZE= and WORKSIZE= options in the PROC IML statement give you control over the size of symbol space and the size of each extent of workspace. If you do not specify these options, PROC IML uses host-dependent defaults. For example, you can begin an IML session and set the SYMSIZE= and WORKSIZE= options with the statement

   proc iml symsize=n1 worksize=n2;

where n1 and n2 are specified in kilobytes.

If the symbol space memory becomes exhausted, more memory is automatically acquired. The symbol space is stable memory and is not compressible like workspace. Symbol space is recycled whenever possible for reuse as the same type of object. For example, temporary symbols can be deleted after they are used in evaluating an expression. The symbol space formerly used by these temporaries is added to a list of free symbol-table nodes. When allocating temporary variables to evaluate another expression, IML looks for symbol-table nodes in this list first before consuming unused symbol space.

Workspace is compressible memory. Workspace fills up as more matrices are defined by operations. Holes in extents appear as you free matrices or as IML frees temporary intermediate results. When an extent fills up, compression reclaims the holes that have appeared in the extent. If compression does not reclaim enough memory for the current allocation, IML allocates a new extent. This procedure results in the existence of a list of extents, each of which contains a mixture of active memory and holes of unused memory. There is always a current extent, the one in which the last allocation was made.

For a new allocation, the search for free space begins in the current extent and proceeds around the extent list until finding enough memory or returning to the current extent. If the search returns to the current extent, IML begins a second transversal of the extent list, compressing each extent until either finding sufficient memory or returning to the current extent. If the second search returns to the current extent, IML opens a new extent and makes it the current one.

If the SAS System cannot provide enough memory to open a new extent with the full extent size, IML repeatedly reduces its request by 2K. In this case, the successfully opened extent is smaller than the standard size.

If a single allocation is larger than the standard extent size, IML requests an allocation large enough to hold the matrix.

The WORKSIZE= and SYMSIZE= options offer tools for tuning memory usage. For data-intensive applications that involve a few large matrices, use a high WORKSIZE= value and a low SYMSIZE= value. For symbol-intensive applications that involve many matrices, perhaps through the use of many IML modules, use a high SYMSIZE= value.

You can use the SHOW SPACE command to display the current status of IML memory usage. This command also lists the total number of compressions done on all extents.

Setting the DETAILS option in the RESET command prints messages in the output file when IML compresses an extent, opens a new extent, allocates a large object, or acquires more symbol space. These messages can be useful because these actions normally occur without the user’s knowledge. The information can be used to tune WORKSIZE= and SYMSIZE= values for an application. However, the default WORKSIZE= and SYMSIZE= values should be appropriate in most applications.

Do not specify a very large value in the WORKSIZE= and SYMSIZE= options unless absolutely necessary. Many of the native functions and all of the DATA step functions used are dynamically loaded at execution time. If you use a large amount of the memory for symbol space and workspace, there might not be enough remaining to load these functions, resulting in the error message

  Unable to load module module-name.


Should you run into this problem, issue a SHOW SPACE command to examine current usage. You might be able to adjust the SYMSIZE= or WORKSIZE= values.

The amount of memory your system can provide depends on the capacity of your computer and on the products installed. The following techniques for efficient memory use are recommended when memory is at a premium:

Free matrices as they are no longer needed by using the FREE command.

Store matrices you will need later in external library storage by using the STORE command, and then FREE their values. You can restore the matrices later by using the LOAD command. See Chapter 17.

Plan your work to use smaller matrices.

17
harlon1976 发表于 2012-4-21 07:18:15
bobguy 发表于 2012-4-21 06:30
Take this first if it helps.

*****************************
感谢你的回答,这个部分是IML中的帮助部分,我也看了,我也使用了symsize语句进行了设置,也使用了free语句,但问题还是不能得到解决,恳请你下载我的程序,然后运行看看,有什么解决的办法。

18
不再躁动 发表于 2012-4-21 09:35:25
harlon1976 发表于 2012-4-20 09:35
能否说得详细点,我觉得应该有解决的办法,请赐教!
根据我的经验 在初始化时候把需要的大矩阵都初始化 就是类似x=zeros(n,m)这样可以节省很多内存 这是和matlab内部处理机制相关的  还有就是用稀疏矩阵 这个要结合实际情况了

19
不再躁动 发表于 2012-4-21 09:37:17
harlon1976 发表于 2012-4-20 09:35
能否说得详细点,我觉得应该有解决的办法,请赐教!
可以把程序发给我 一块分析分析 一起研究下 哈

20
harlon1976 发表于 2012-4-21 11:15:50
不再躁动 发表于 2012-4-21 09:37
可以把程序发给我 一块分析分析 一起研究下 哈
程序已经附上了,你自己下载啊!

您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加好友,备注jltj
拉您入交流群
GMT+8, 2026-1-2 10:24