This error message usually appears when there is a datatype mismatch defined within a Formula. An example of this is where the result of the Formula is a string but the target field you are trying to update is of type Whole Number.
The first resolution of this issue is review the formulas you have defined for the entity that threw the error message and try to find a datatype mismatch between the result of a formula & its target property.
The next resolution of this issue is enabled Dynamics CRM 2011's native tracing. The following KB article from Microsoft explains how to do this.