Using the Crossvalidation service
Service procedure
The crossValidation (xval) process splits the sample data into subsets such that the analysis is initially performed in a single subset, while the other subset(s) are retained for subsequent use in confirming and validating the initial analysis.In K-fold xval the original data is partitioned into K subsamples. The crossvalidation is then repeated K times. If K == number of observations we have a Leave-One-Out CrossValidation 'LOOCV', where a single observations from the original data is removed and its value estimated using the remaning set. The xval process has a default K-fold of 10. The use of LOOCV is not advised since it is extremely time consuming
The xval process is designed to use the same inputs/outpus of the INTAMAP system. Therefore it uses UncertML and Obs&Meas as major XML structures. The xval service runs as a complete python script integrated in a pyWPS implementation, therefore it is not possible to port it to another system architecture as an API (like the INTAMAP-System). This the xval service can only exist as a web service
Currently a xval service is available at:
http://remwps2.jrc.ec.europa.eu/cgi-bin/wps.pyInput
Several xval request examples can be found in the SVN tree.The examples are based on the Meuse data set and INTAMAP-System interpolation requests Using the command wget it is possible to send a request and obtain a WPS response with the result or the status document: wget -O - -n -q --post-file=
Basic input
Following the INTAMAP philosophy, the xval basic input is minimal, a simple data set as Obs&Meas in the XML request will generate a response. In this case the service will use all the automtic settings located in the interpolation service, the interpolation model will be picked automatically and it will use the default k-fold 10 for crossvalidationComplete list of possible inputs:
- ObervationCollection (mandatory as Obs&Meas)
- Kfold
- InterpolationServer
- ProcessName
- MethodName
- MethodParameters
- MaxTime