I built a system to collect and display social recommendations for HBO shows. It had three main parts: a crawler that monitored Twitter for mentions of HBO shows, an automated tagger that used natural language processing to categorize these mentions, and a content management system where HBO staff could review and publish the best recommendations.
The twitter crawler was consuming the twitter api based on predefined keywords and mentions. It was implemented as a scheduled Lambda function that would then feed the content into an SQS queue.
The content from the queue was then processed by an additional lambda function for sentiment analysis, categorization and simple moderation. The results were then stored in a DynamoDB table for further processing.
The processed content would then be fed into a Contentful CMS for the HBO team to curate and eventually publish on the site