请选择 进入手机版 | 继续访问电脑版
楼主: wyxixl
6521 28

[原创博文] 问题已解决!希望能置顶!如何让SAS9.2 能够将处理器的多核心都用起来 [推广有奖]

wyxixl 发表于 2011-11-19 21:30:58 |显示全部楼层 |坛友微信交流群
上面的这位兄弟的方法我试过了,默认设置就是CPU1和CPU0,我重新运行了SAS,满负荷运行时,任务管理器中SAS进程的CPU占用最大不会超过50%。所以这个方法不灵的。还希望有哪位能提供真正的解决方案,或者启示?

使用道具

wyxixl 发表于 2011-11-23 16:58:04 |显示全部楼层 |坛友微信交流群
这么好的帖子没人顶,自己顶一个!

使用道具

fxf258 发表于 2011-11-23 17:14:22 |显示全部楼层 |坛友微信交流群
我也顶一下,希望引起大家的重视!
海象

使用道具

lyceum 发表于 2011-11-24 08:32:06 |显示全部楼层 |坛友微信交流群
试试这个 OPTIONS THREADS=YES CPUCOUNT=
把你所有CPU数目填在等号右边,比如你有两个CPU就用 OPTIONS THREADS=YES CPUCOUNT=2

使用道具

lyceum 发表于 2011-11-24 08:33:05 |显示全部楼层 |坛友微信交流群
试试这个 OPTIONS THREADS=YES CPUCOUNT=
把你所有CPU数目填在等号右边,比如你有两个CPU就用 OPTIONS THREADS=YES CPUCOUNT=2

使用道具

wyxixl 发表于 2011-11-26 08:59:58 |显示全部楼层 |坛友微信交流群
楼上这位兄弟的方法我试过了,还是不行,任务管理器里面SAS的CPU占用率始终无法超越50%。是不是SID种子不够新?有关模块没打开?不过还是谢谢你了!

使用道具

wyxixl 发表于 2011-11-26 09:03:23 |显示全部楼层 |坛友微信交流群
到了SAS 9版本,是个革命性的版本,它提供了利用多CPU和多I/O的多线程任务计算,只需要在SAS或者不同的proc步中定义thread选项就可以实现高效的多线程运算。比如SORT, SUMMARY, REG, and GLM等proc步与SAS/Access都是该技术的受益者
楼上的方法是不是只能在某些特定的 proc步中才能使用?

使用道具

wyxixl 发表于 2011-11-26 16:49:54 |显示全部楼层 |坛友微信交流群
Re: Sas/Windows Cpu Utilization Does Not Increase Beyond 50%
I just ran some tests and even if CPU is running at 100% it still seems the
threading overhead and i/o go some way to offset any benefits. If you have a
pure cpu operation then it should benefit. Having said that, threads=YES and
cpucount=2 (or whatever is your maximum) consistently came near the top in
performance times. Addmittedly at the lower end times aren't going to make
any appreciable difference, and I found where it began to in the last test
it was at the point I began to have virtual memory issues, but by this point
the tYESc2 advantage does become very significant.

I agree with Alan on a PC you've more to gain from a more holistic approach
to performance tuning than merely wanting proc sort to max out your
processor (I always use windows compression AND compress=YES). Although I
expect on a multi-cpu server this sort of performance fine tuning may start
to yield clearer advantages.

Rgds

options threads=YES CPUCOUNT=2;
proc sort data = temp1 out=_NULL_;by isn;run;
/*
1800k obs (sort var only)
real time 4.28 seconds
cpu time 5.43 seconds
3600k obs (sort var only)
real time 8.93 seconds
cpu time 11.01 seconds
3600k obs (sort var only null output)
real time 3.51 seconds
cpu time 5.82 seconds
7200k obs (sort var only null output)
real time 14.42 seconds
cpu time 15.62 seconds
14500k obs (sort var only null output)
real time 36.51 seconds
cpu time 30.90 seconds

*/

options threads=YES CPUCOUNT=1;
proc sort data = temp1 out=_NULL_;by isn;run;
/*
1800k obs (sort var only)
real time 4.37 seconds
cpu time 4.12 second
3600k obs (sort var only)
real time 9.03 seconds
cpu time 8.62 seconds
3600k obs (sort var only null output)
real time 4.40 seconds
cpu time 4.43 seconds
7200k obs (sort var only null output)
real time 24.17 seconds
cpu time 12.25 seconds
14500k obs (sort var only null output)
real time 1:12.51
cpu time 26.96 seconds

*/

options threads=NO CPUCOUNT=1;
proc sort data = temp1 out=_NULL_;by isn;run;
/*
1800k obs (sort var only)
real time 4.17 seconds
cpu time 4.00 seconds
3600k obs (sort var only)
real time 8.98 seconds
cpu time 8.36 seconds
3600k obs (sort var only null output)
real time 3.89 seconds
cpu time 3.87 seconds
7200k obs (sort var only null output)
real time 26.78 seconds
cpu time 12.18 seconds
14500k obs (sort var only null output)
real time 1:07.64
cpu time 26.01 seconds

*/

options threads=NO CPUCOUNT=2;
proc sort data = temp1 out=_NULL_;by isn;run;
/*
1800k obs (sort var only)
real time 4.25 seconds
cpu time 4.03 seconds
3600k obs (sort var only)
real time 9.00 seconds
cpu time 8.48 seconds
3600k obs (sort var only null output)
real time 3.85 seconds
cpu time 3.82 seconds
7200k obs (sort var only null output)
real time 27.18 seconds
cpu time 12.04 seconds
14500k obs (sort var only null output)
real time 1:06.12
cpu time 25.97 seconds

*/

使用道具

wyxixl 发表于 2011-11-26 16:50:46 |显示全部楼层 |坛友微信交流群
问题的真相快解决了!

使用道具

wyxixl 发表于 2011-11-26 16:56:57 |显示全部楼层 |坛友微信交流群
感谢各位的无私帮助,在lyceum的提示下,我已经找到正确答案,现在免费提供给大家,希望版主能划转50金币给lyceum同学
,谢谢!请各位转帖时,请注明来自本帖。


Paper CC-16
Use Your Cores! An Introduction to Multi-core Processing with SAS
Erik Dilts, INC Research, Raleigh, NC
ABSTRACT
Computing power has changed direction the past few years. In the past, chip manufacturers designed faster and faster processors (CPUs), but they reached the theoretical limit of that method. This is how the multiple-core CPU in today’s computers came to be. A multi-core CPU is essentially the same as having multiple CPUs. SAS® 9 can take advantage of these cores in several different ways. This paper will explain how to use multi-core processing to your advantage. In addition it will demonstrate the pros and cons of the various techniques available under Windows, and when to use them.
INTRODUCTION
Multi-core processing can save a great deal of time when running large jobs or a large number of jobs. Due to the overhead involved in coordinating the flow of data between the cores, there is rarely a 1:1 ratio in the amount of time a program takes to run on multiple cores. That is, two cores will not make a job run twice as fast as one core. Some techniques for taking advantage of multiple cores require no changes at all to existing code, while others are relatively complex and require extensive modification. When trying to split one single job up among multiple cores, the timing of program submission and completion is critical. Since SAS® is a procedural language, one step must finish before the next one can proceed. When running multiple jobs simultaneously, the only timing that needs to be considered is program dependency, i.e., if program 2 depends on the completed output from program 1. If your programs have no such dependencies, then running multiple jobs at once becomes very simple.
SETUP
The simplest way to use multiple cores is with an option: THREADS. This can be done at invocation or in an options statement. Many SAS® processes will then “thread” their operations over multiple cores. Some of the procedures that can do this are SORT, MEANS, SQL, and GLM. A complete list can be found in the online SAS® Language Reference Dictionary. This method does not require any changes to existing code.
The next method of using multiple cores is multiple batch submissions. This is the standard format to batch submit two SAS® jobs:
START/W SAS.EXE –SYSIN “PROGRAM1.SAS”
START/W SAS.EXE –SYSIN “PROGRAM2.SAS”
The “/w” switch tells Windows to wait until the first step is finished before going to the next. To submit both programs at once the following format should be used:
START SAS.EXE –RSASUSER –SYSIN “PROGRAM1.SAS”
START SAS.EXE –RSASUSER –SYSIN “PROGRAM2.SAS”
Now Windows will submit both jobs immediately. The “–rsasuser” option tells SAS® to open your profile in read-only mode so as not to lock it (which generates a log warning when multiple sessions are open at once). This method also does not require changes to your existing SAS® code. The one caveat for this method is program dependency. If Program2 requires the output from Program1, then obviously they cannot be submitted at the same time. There is another consideration for this method. If you have 20 programs you want to run, and you do not use the /w switch for any of them, then all 20 will be submitted at once – and unless you have 20 cores, that won’t work very well and the jobs might even run slower. This is where things get a little more complex, but one solution is to create multiple batch programs, and then use another batch program to call them one after the other.
In general you should only have one job running per core. You can find out how many cores you have via the Windows Task Manager, which is invoked using the Ctrl-Alt-Del key combination.
2
This is how the performance tab looks on my laptop:
The split in the CPU Usage History graph indicates that I have two cores available.
The last method I will discuss is the use of RSUBMIT within one SAS® program to submit multiple parts of it at once. This is custom programming – it is not always straightforward to modify existing code to do this. For example, you might have a very large dataset with male and female subjects, and need to analyze it with PROC MEANS. You could increase the efficiency of the program by submitting each group at the same time.
SIGNON TASK1 SASCMD="!SASCMD" INHERITLIB=(WORK=LOCWORK); * create server session 1;
RSUBMIT TASK1 WAIT=NO;
PROC MEANS DATA=DATASET;
WHERE GENDER=’M’;
OUTPUT OUT=MEANS_M;
RUN;
ENDRSUBMIT;
SIGNON TASK2 SASCMD="!SASCMD" INHERITLIB=(WORK=LOCWORK); * create server session 2;
RSUBMIT TASK2 WAIT=NO;
PROC MEANS DATA=DATASET;
WHERE GENDER=’F’;
OUTPUT OUT=MEANS_F;
RUN;
ENDRSUBMIT;
WAITFOR _ALL_ TASK1 TASK2; * wait for both tasks to complete before continuing;
DATA MEANSOUT;
SET MEANS_M MEANS_F;
RUN;
One thing to consider when using this method is that, even though it is just one SAS® session that is invoking others, they are still separate sessions, which means that parameters like options, macro variables, and library locations must be passed to each of them – they are not automatically inherited from the calling session. Timing also needs to be considered. SAS® must know that the two procedures both have to finish running before the data step runs.
CONCLUSION
SAS ® version 9 has many enhancements to take advantage of the multiple cores that exist in modern computer processors. Some of these enhancements require little to no changes to existing code, while others require that programs are written in a very specific manner. Chances are, the “extra” cores on your computer are not even being used, at least not specifically by SAS®. Multiple-core processing can greatly increase the efficiency of your SAS® jobs so you can get more work done in less time.
CONTACT INFORMATION
Your comments and questions are valued and encouraged. Contact the author at:
Erik Dilts
INC Research
4800 Falls of Neuse Rd., Suite 500
Raleigh, NC 27609
Work Phone: 919-334-3532
E-mail: edilts@incresearch.com
SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.
Other brand and product names are trademarks of their respective companies.

使用道具

您需要登录后才可以回帖 登录 | 我要注册

本版微信群
加好友,备注cda
拉您进交流群

京ICP备16021002-2号 京B2-20170662号 京公网安备 11010802022788号 论坛法律顾问:王进律师 知识产权保护声明   免责及隐私声明

GMT+8, 2024-3-29 10:37