Skip to content

A topic modeling pipeline using the BERTopic model and state-of-the-art technologies

License

Notifications You must be signed in to change notification settings

AndrewDarnall/TAP_Project

Repository files navigation

Project Logo

The Observer

The goal of the project is to provide a 'no strings attatched' overview of public mastodon servers leveraging the strength of the BERTopic model and the extensible architecture ofthe project


Architecture

Miro board here

Architecture Diagram


Setup

  • It is preferable to run the project in a Linux environment or use wsl
  • It is advised to run this project wither on the cloud, with a powerful VM instance or with at least 16GB or RAM and Linux
  1. clone the directory with the git cli tool
  2. chdir into the cloned directory
  3. bash project_setup.sh
  4. docker-compose build (This might take a while)

Run

  1. docker-compose up -d (If you have limited hardware, it might take a while)

Configure Dashboard

  1. in your local browser, browse http://localhost:5601/
  2. Go to > Saved Objects > Import > TAP_Project/Data_Visualization/saved-objects/dashboard_export.ndjson > click import
  3. Reload the page as is
  4. Go to dashboard and select the 'the_observer' dashboard

Notes

Do with the project as you please, I designed it to be expandable and hihgly maintainable, therefore you can add other servers, other social medias so longs as you maintain the .jsonl format in the datastorage volume

End Result

Dashboard One


Dashboard Two

About

A topic modeling pipeline using the BERTopic model and state-of-the-art technologies

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages