We present findings of a study with 62 subjects who had 796 of their outgoing mobile phone calls recorded and transcribed for their later annotation—by highlighting important information shared during calls. We found that patterns in these calls (numbers, names, interrogative adverbs), as well as some contextual parameters, are better indicators of annotation needs than the callers' profile or call quality. Callers highlight information in both parties' turns (caller and callee) more often than highlighting solely information provided by the callee, which is mostly due to annotating questions with contextual information for the highlights in the callee's turns. We discuss how this behavior changes according to call purpose. Finally, we found that annotation needs change over time: whereas some annotations might not be considered relevant after weeks, others originally considered irrelevant might become important archival notes. We present implications of these findings for the design of mobile phone annotation tools.