Java (JPMML) Prediction using R PMML model (Part 2)

Last post showed how to generate a linear regression PMML model using R. This blog will load this PMML model into Java, via the excellent JPMML package, and perform predictions using a streaming test data set.

The method below represents how a PMML file is loaded in to memory using the JPMML library. Once the PMML has been load we are ready to perform prediction using streaming data. In this example our stream is a test csv file but in a production environment this can be real-time streams or some offline prediction mechanism using captured data ready for a start of day process.

/**
 * Load a PMML model from the file system.
 *
 * @param file
 * @return PMML
 * @throws Exception
 */
public final static PMML loadModel(final String file) throws Exception {

   PMML pmml = null;

   File inputFilePath = new File( file );

   try( InputStream in = new FileInputStream( inputFilePath ) ){

     Source source = ImportFilter.apply(new InputSource(in));
     pmml = JAXBUtil.unmarshalPMML(source);

   } catch( Exception e) {
      logger.error( e.toString() );
      throw e;
   }
   return pmml;
}

Now we have loaded the linear regression model as a PMML object we can start the real work of setting up the prediction processing. The next code section shows how to load, gain a predictive evaluator and get the list of input feature fields needed for the evaluator.

// Load the file using our simple util function.
PMML pmml = JPMMLUtils.loadModel( pmmlFilePath );

// Now we need a prediction evaluator for the loaded model
PMMLManager mgr = new PMMLManager( pmml );

ModelEvaluator modelEvaluator = (ModelEvaluator) mgr.getModelManager(modelName, ModelEvaluatorFactory.getInstance());
Evaluator evaluator = modelEvaluator;

// Get the list of required feature set model needs to predict.
List requiredModelFeatures = evaluator.getActiveFields();

Below is the main body of the prediction routine. The code performs the following steps:

1. Parse the passed csv line into tokens and split into required variables.
2. Get the required feature variable field name.
3. Prepare the field to value mapping and assign to feature hashmap.
4. Perform the prediction (evaluate function).
5. Convert values back to original state and print out result.

 // For each CSV line perform a predict.
 while ((line = br.readLine()) != null) {

     String[] tokens = line.split( cvsSplitBy );
     
     double d =  Double.valueOf( tokens[2] );
     double e = Double.valueOf( tokens[3] );

     FieldName fieldName = requiredModelFeatures.get(0);

     // In this instance I know there is only one feature
     // For a production system this would be performed in a transformation stage and may collect data externally.
     FieldValue value = evaluator.prepare(fieldName, Double.valueOf(d));
     features.put( fieldName, value );

     Map results = evaluator.evaluate( features );

     // Convert back to original ring value so the prediction become meaningful.
     double y = (Double)results.get( evaluator.getTargetField());
     int predictedRings = (int) Math.abs( Math.pow( 10, y));

     int expectedRings = (int) Math.abs( Math.pow( 10, e));

     double diameter =  Math.pow( 10, d);
     System.out.println(String.format("Diameter %f - Expected rings %d : Predicted rings: %d", diameter, expectedRings, predictedRings));
}

The section below presents the predictions performed using the JPMML linear regression evaluator using the passed offline R model. As you can see they are not perfect but the algorithm is generalizing reasonable well give the minor effort placed to build the model.

    Diameter 0.225000 - Expected rings 7 : Predicted rings: 6
    Diameter 0.530000 - Expected rings 11 : Predicted rings: 11
    Diameter 0.500000 - Expected rings 11 : Predicted rings: 11
    Diameter 0.460000 - Expected rings 8 : Predicted rings: 10
    Diameter 0.270000 - Expected rings 7 : Predicted rings: 7
    Diameter 0.510000 - Expected rings 11 : Predicted rings: 11
    Diameter 0.295000 - Expected rings 10 : Predicted rings: 7
    Diameter 0.450000 - Expected rings 9 : Predicted rings: 10
    Diameter 0.350000 - Expected rings 7 : Predicted rings: 8
    Diameter 0.195000 - Expected rings 6 : Predicted rings: 5
    Diameter 0.435000 - Expected rings 18 : Predicted rings: 10
    Diameter 0.410000 - Expected rings 11 : Predicted rings: 9
    Diameter 0.455000 - Expected rings 9 : Predicted rings: 10
    Diameter 0.440000 - Expected rings 11 : Predicted rings: 10
    Diameter 0.270000 - Expected rings 7 : Predicted rings: 7
    Diameter 0.495000 - Expected rings 11 : Predicted rings: 11
    Diameter 0.525000 - Expected rings 11 : Predicted rings: 11
    Diameter 0.205000 - Expected rings 3 : Predicted rings: 5
    Diameter 0.510000 - Expected rings 10 : Predicted rings: 11
    Diameter 0.375000 - Expected rings 8 : Predicted rings: 9
    Diameter 0.135000 - Expected rings 5 : Predicted rings: 4
    Diameter 0.465000 - Expected rings 11 : Predicted rings: 10

This now concludes the R to Java PMML integration sessions. My next algorithm I will be looking at is k-NN using the same technique. If you require any further information on how I did this please email me directly.

Linear Regression in R to PMML (Part 1)

As a part of my efforts to increase my understanding of machine learning algorithms I am putting together a set of blogs to experiment algorithm usage. My chosen languages are R, Java and Python to implement, train and test algorithms. I am very much a novice in the area of R but very much the opposite in Java, so any feedback is most welcome.

Models will be created in R and will be exported using Predictive Model Markup Language (PMML). This enables cross language support for ML algorithms while providing the platform to perform deep offline learning using Big Data solutions, SAS and GreenPlum DB are good platform examples to investigate.

The first ML algorithm I shall start with is the most basic; linear regression. This project I shall be using the famous abalone dataset to predict the number of rings an abalone would have based on a set of chosen features. To acheive this I shall be using R to train and validate the model, PMML model export and a Java JPMML example to read the resulting offline model to predict number of abalone rings from the held out test set. So lets get started.

Data Preparation stage
First load the data, add the column names, select required features and scale only those features. In this instance the data does not need to be cleaned.

require(pmml)     # Exporting the model in xml
require(ggplot2)  # Visualization

# Load the data
abalone <- read.table("../data/abalone.csv",sep=",", header=TRUE)
abaloneNames <- c("sex","length","diameter","height","whole_weight","shucked_weight","viscera_weight","shell_weight","rings")
colnames(abalone) <- abaloneNames

# Remove Sex column.
abalone$Sex <- NULL

# Select the feature space
features <- c("whole_weight", "diameter", "rings", "length", "height")
abalone <- abalone[ features ]

# Move features in to log space since lm works better when all features are equally scaled. 
abalone <- scale( abalone )

# Now split the dataset in to train 80%  and test 20%
indexes <- sample(1:nrow(abalone), size=0.2*nrow(abalone))
test <- abalone[indexes,]
train <- abalone[-indexes,]

Training stage
Now start the training process. In this instance I am only going to use a single feature as an indicator to the number of rings. The resulting graph provides a sense how the prediction performs by plotting the training data against linear separator.

# Now train only using a single feature
ringsmodel <- lm(formula = rings ~ diameter, data = train)

# Make sure the p-value is less than 5%
summary( ringsmodel)

Call:
lm(formula = rings ~ diameter, data = train)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.25960 -0.06873 -0.01987  0.05219  0.40065 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 1.284061   0.005876  218.52   <2e-16 ***
diameter    0.761835   0.013837   55.06   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.1005 on 3340 degrees of freedom
Multiple R-squared: 0.4758, Adjusted R-squared: 0.4756 
F-statistic:  3031 on 1 and 3340 DF,  p-value: < 2.2e-16 


# Get a sense how the prediction performed
qplot(x = diameter, 
      y = rings,
      data = train,
      alpha = I(0.2), # alpha makes the points semitransparent so you can see stacked points
      geom = "jitter") + # jitter helps spread the points so they don't stack so much
  geom_smooth(method = lm)

At this point we could iterate the training process by adding in additional features hoping to reduce p-value further. However, given this is low enough for this example I will stop here. As a side point a p-value less than 5% is good enough for predictive generalisation.

Test the prediction to actual outcomes stage
Now we are ready to test how out model performs using the test data we put aside earlier. To help review the I have performed a join with the predicted data with the actual data along with the error.

# Use the test data against the predictive model.
p <- predict.lm( ringsmodel, test, se.fit=TRUE )
pred.w.plim <- predict( ringsmodel, test, interval="prediction")
pred.w.clim <- predict( ringsmodel, test, interval="confidence")
matplot(test$diameter,cbind(pred.w.clim, pred.w.plim[,-1]),
        lty=c(1,2,2,3,3), type="l", ylab="predicted y")


# Join the actual with the predicted rings
prediction <- data.frame(actual = test$rings,  predicted = p$fit, error = test$rings - p$fit )

As you can see the result is reasonable enough to suggest the model has not overfitted the data and is generalising reasonable well.

Export the final model stage
The final part of this process is to export the model to pmml for a consuming application to predict with. The next blog with read the xml file using JPMML and perform the same tests used above.

# Export the resulting model as PMML file.
localfilename <- "../models/abalone-rings-lm-prediction.xml"
saveXML(pmml( ringsmodel, model.name = "AbaloneRingsPredictionLM", app.name = "RR/PMML", dataset = dataset) , file = localfilename)

# Save the test data for later use as csv
write.csv(test, file = "../data/test_abalone.csv")