CPW: Class and Prototype Weights learning
In order to optimize the accuracy of the Nearest-Neighbor classification rule, a weighted distance is proposed, along with algorithms to automatically learn the corresponding weights. These weights may be specific for each class and feature, for each individual prototype, or for both. The learning algorithms are derived by (approximately) minimizing the Leaving-One-Out classification error of the given training set.
Reference
For all the details please read the draft copy of the paper CPW_Paredes_Vidal-PAMI2006.pdf
Source code
- CPW C - implementation (and Nearest Neighbor with Weighted Distance classifier for testing the results)
Changelog
- Initial version
- Please report bugs to rparedes@dsic.upv.es
Corpus for testing
- Australian:Training and Test
- Balance:Training and Test
- Diabetes:Training and Test
- DNA:Training and Test
The files have as many rows as prototypes. Each row is a d-dimensional vector (prototype) and the last column is the class label (it can be a string) so each row has d+1 columns.
Help
1. Compile (Linux):
gcc -o cpw cpw.c -O6 -lm
gcc -o nncpw nncpw.c -O6 -lm
2. Learning:
cpw trainingfile [-sigm slope] [-mu mu] [-nu nu] [-thr thres] [-i it] [-v] [-V]3. Optional Parameters:
| -sigm | Sigmoid slope (10) |
| -mu | Class dependent feature weights Learning Step (0.001) |
| -nu | Prototype Weight Learning Step (0.0, only feature weights) |
| -thr | Threshold to stop iterations (0.000001) |
| -i | Max Iterations (1000) |
| -v | Use class-dependent mahalanobis to initialize the feature weights |
| -V | Verbose |
4. Some examples:
- Basic use:
cpw balance_training
- Use class-dependent mahalanobis for initialization
cpw aus_training -v
- To learn also prototypes weights
cpw diabetes_training -nu 0.01
- To learn ONLY prototypes weights
cpw diabetes_training -mu 0.0 -nu 0.01
- To learn ONLY prototypes weights BUT using Mahalanobis as "basic" distance
cpw diabetes_training -v -mu 0.0 -nu 0.01
5. Program result, two files:
- cweights.cpw: file with the class-dependent weights
- pweights.cpw: file with the prototype weights
6. Test the result:
nncpw aus_training aus_test cpweights.cpw pweights.cpw
if not pweights file is provided the program assume 1.0 for all the prototypes
cweights.cpw and pweights.cpw must be the weights learned by the training set used for test