I'm trying to develop a model in R that will compare a customer review with a summary of that review that is completed by an employee. The purpose is to ensure that the employee is accurately tagging and summarizing the customer review.
In more detail:
- A customer leaves a review
- Employee reads the review and creates a summary of the review
- Employee creates tags such - technical problem, billing issue, etc.
Which NLP method would be best to compare the review with each summary? Is this something that's even possible?
I don't think Bag-of-Words would be useful. I understand that doc2vec can be used compare texts and find the texts that are the most similar to one another. However, how do I measure how accurately a summary reflects its original review?
Say I have 1000 reviews: Is the only way to do this if I:
- Manually read 500 reviews and summaries, and tag each summary as 1=good summary or 2 = bad summary.
- Train the model using some NLP tool.
- Test the model on the remaining 500 reviews and summaries?
I do not have sample data at this time. I am only doing literature review to see if this is possible.
Thanks!
