What you've described so far is the start of one cross-validation step. Here's the generic procedure:
1) Divide data set at random into training and test sets.
2) Fit model on training set.
3) Test model on test set.
4) Compute and save fit statistic using test data (step 3).
5) Repeat 1 - 4 several times, then average results of all step 4.
Cross-validation is one method of trying to reduce overfitting (optimism) in a fitted model. Typically these are regression-based models used for prediction. By randomly dividing the data set as above, there is less certainly about the final model, but on aggregate, the process tells you something about how the model might generalize to a new independent data set. This is one way of performing model validation.
There are several types of cross-validation that can be broadly divided into "exhaustive" or "non-exhaustive" methods. Exhaustive methods mean that every possible training/test split is performed. These are the leave-one-out or leave-n-out methods in which one data point (or n points) are used for the test set. The non-exhaustive methods divide the data into equally sized groups (k-fold cross-validation) or use more complex methods to sample the data.
The measure you should use for binary classification depend on what you are trying to do, and there are lots of measures of fit that might be of interest (mean absolute deviation, Brier score, c-statistic, $R^2$, positive/negative predictive value, precision, recall). You'll need to look at related literature or tell is more information to decide which of these you will need.
The cross-validation procedure only tells you about the degree of overfitting (optimism) in the original model. Once it's done, the model validation has been characterized. It's up to you to determine whether the amount of overfitting (optimism) is acceptable or not using existing literature or expert judgement. Model validity in this sense only means the model makes sense and has an acceptable level of error, not that it is the perfect or best model (which is most likely impossible to ever know).
