Jon Peck:Important SPSS Updates

Three Statistics extension commands have been updated.

SPSSINC RANFOR and SPSSINC RANPRED, which produce and predict using random forests, have been rewritten so that they no longer require the Python Essentials.  They now require only the R Essentials.  They are repackaged to use the standard extension bundle installer on the Utilities menu, and they have some new features and improved output.

SPSSINC APRIORI, which builds association rules, has similarly been rewritten to require only the R Essentials and to use the extension bundle installer .  It also has a few new features.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-562

As has already been discovered, fixpack1 for Statistics V21 is now available: it was posted sometime Wednesday.  While we like to keep the list informed, this is an informal mechanism and shouldn't be relied on as the official notification mechanism.

I would like to remind users that we welcome contributed code on the SPSS Community site.  Anyone can post something to their (free) account on IBM developerworks - www.ibm.com/developerworks/mydeveloperworks - subject to the reasonable Terms & Conditions posted on the site (no copyright or patent infringement, nothing malicious etc).  If you post a file and would like it listed in one or more of the SPSS Community collections, that is easy to  arrange.  Instructions are on the Welcome page of the site.

If you set up an account, you can also create your own blog on dW.  The SPSS Community blog is not as active as it should be, but the bookmarks on the site list a number of other blog sites with good content, such as Andy W's.  If you know of other good blogs that should be listed, you can send me a note.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

A pair of new extension commands for Statistics have been posted to the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral)

STATS GBM estimates generalized boosted regression models including regular regression, logistic, multinomial, poisson, and quantile regression among others.
STATS GBMPRED uses the estimates from STATS GBM to calculate predicted values for new data.

These procedures use the R gbm package and require the R Essentials.  They were developed using Statistics V21 but should work with version 18 or newer.

I am calling these beta versions as there are many features that are difficult to test comprehensively in combination.  Please report any bugs or comments to me.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

The STATS LATENT CLASS extension command has been updated to allow you to save a dataset of scores, i.e., the predicted class, for each input case.  The dataset contains an ID, the maximum probability class, and the probabilities for all the classes.  The procedure can be found in the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral).

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

A new extension command, STATS ADJUST WIDTHS, has been added to the Extension Commands collection on the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) under Downloads for SPSS Statistics.  This command synchronizes the widths of selected string variables across different sav files or datasets in order to facilitate file merging and similar activities.  It provides several choices on how to set the widths and various options about what to do with the modified files.  The command requires the Python Essentials and should work with V18 and above, although it has been tested mainly on V21.

Also, I forget to mention in the last news note that the extension.py file installed with the Python Extensions has been updated to address a problem that appears sporadically with some combination of Windows 8 and 64-bit Statistics.  If you have not seen a mysterious error emanating from this module you don't need this update.  If you have, you can get the update from the Python modules collection on the Community site.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

A new extension command, STATS LOESS, has been added to the Extension Commands collection on the SPSS Community site (www.ibm.com/developerworks/spssdevcentral).  It fits a loess regression curve for a model with 1-4 independent variables and saves the predicted values for the estimation data or new data values to a new Statistics dataset.

This command requires the R Essentials and at least Statistics version 18.

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-56

A new extension command, STATS CARTPROD, has been uploaded to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.  It computes the Cartesian product of two sets of variables in the same or separate datasets and writes the result to a sav file and opens it.  While not as fast as this operation would be in a database, it is quite general and preserves all variable properties.  (All the data operations are carried out with traditional Statistics syntax, but Python code is used to generate that syntax.)

This command requires the Python Essentials.  The Essentials can be obtained via the Community site or the Statistics product materials, depending on the Statistics version.  The extension command can be downloaded from the Extension Commands collection or installed directly using Utilities > Download and Install Extension Bundles with Statistics version 22.

Regards,


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621

One new item and one updated item have been posted to the SPSS Community website (www.ibm.com/developerworks/spssdevcentral).

The new item: Apropos of the recent discussions on this list about propensity score matching, a new custom dialog box has been uploaded to the Statistical Tools Collection on the site to simplify doing PSM.  It does a logistic regression following by matching and produces a new dataset of cases and their matched controls.

PSM requires at least version 1.3.0 of the FUZZY extension command.  That version of FUZZY is included in the Python Essentials for Statistics 21 and can also be downloaded from the Community site in the Extension Commands Collection.  PSM has been tested only with Statistics version 21 but may also work with versions 18-20.

The updated item: The version of the FUZZY extension command that includes a single-dataset dialog box interface in Statistics 21 has a bug in the dialog box that disabled fuzz factors.  That has been fixed in the version now posted in the Extension Commands Collection on the site.  The bug affected only the dialog box.

Regards,

Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

The SPSSINC TRANS extension command, which essentially allows Python functions to be used as transformations, has been updated.  It now provides control over user missing values, and there is a new subcommand that allows the TO construct to be used in variable lists in the FORMULA subcommand.  (TO could already be used in the results list.)

The new version can be downloaded from the SPSS Community site (www.ibm.com/developerworks/spssdevcentral) via the link to Downloads for SPSS Statistics and then to the extension commands collection.

SPSSINC TRANS is now included in the Python Essentials, but the posted version is newer.


Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
new phone: 720-342-5621

In honor of Shakespeare's birthday :-) two new extension commands have been uploaded to the SPSS Community website.

STATS FIRTHLOG estimates the Firth variation of binary logistic regression, which can address the separation issues that sometimes arise in the standard logistic model, especially when the sample size is small.

From the algorithm description...
In general, maximum likelihood estimates are often prone to small sample bias. To reduce this bias, Firth suggested to maximize the penalized log likelihood log L(beta)* = log L(beta) + 1/2 log|I(beta)|, where I(beta) is the Fisher information matrix, i. e. minus the second derivative of the log likelihood. Applying this idea to logistic regression, the score function U(beta) is replaced by the modified score function U(beta)* = U(beta) + a, where a has rth entry a_r = 0.5tr{I(beta)^{-1} [d I(beta)/d beta_r]}, r = 1,...,k.

STATS INEQUALITY calculates up to seven measures of distribution inequality for a variable, including Gini, Atkinson, Theil, and others.  It supports split files, which makes group comparisons easy.

These commands require the R Essentials and at least version 18 of Statistics.  They can be downloaded from within Statistics using the Utilities menu in version 22. For older versions you can download them from the SPSS Community website (www.ibm.com/developerworks/spssdevcentral) in the Extension Commands collection.



Jon Peck (no "h") aka Kim
Senior Software Engineer, IBM
peck@us.ibm.com
phone: 720-342-5621