This refactor focuses on implementing parallelization. This allows us to pass in a number of threads and download concurrently. This makes downloading lots of images a lot faster.
* Weapons can now download the “raw”image size, which is the weapon art without a background
* Characters now download the “detail” image size, which is a horizontal crop of the character’s art
* Summons now download the “detail” image size, which is a horizontal crop of the summon’s art
* Summons also download “ls” and “m” instead of “party_main” and “party_sub”, as they match the aspect ratio of weapon sizes better, which should make our lives a lot easier.
* Update weapon series migration
This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.
* Update .gitignore
Hide backups and logs directories, since we’ll be storing these in the project folder. Also hide mise’s .local directory.
* Change NewRelic log directory
Moved from log/ to logs/
* Add rake task for backing up/restoring prod db
* Rubocop fixes
* Fix error where :preview_state didn’t have an attribute
* Add supervisord ini
This uses my local paths, so we should try to abstract that away later.
* Ignore mise.toml
* Add new items
Items from:
- Unite and Fight (January 2025)
- Story Event (January 2025)
- Negima Collab (February 2025)
- January Flash Gala
- January Legend Festival
- February Flash Gala
- February character uncap
Data fixes for
- Chat Noir FLB (rarity was set to a bad value)
- End of year weapons (FLB date not set)
* Fix CSVs
* Fix data update rake task
* Add CSV require in BaseImporter
* Moved queries into PartyQueryBuilder service
PartyQueryBuilder supersedes PartyQueryingConcern as it is also used for UsersController (and is our fix for profiles being broken)
* Implement PartyQueryBuilder in controllers
* Update summon_transformer.rb
This should fix the transformer so that we properly capture summons and subaura summons
* Update parties_controller_spec.rb
* Add NewRelic license key
* Add Sentry
Why not?
* Add default preview images
* Update application.rb
* Adds app assets path in API mode
* Cleans up file
* Create assets.rb
An initializer for font assets (for image generation)
* Updates to Canvas and Coordinator
* Update parties_controller.rb
* Adds retry header if generation is still in progress
* Streams S3 content instead of redirecting to prevent 302
* Update coordinator.rbs
* Create previews.rake
A rake task for generating images offline
* Add commands to build phase
* Only re-index search when records are updated
* Add logs and fixes
* Add preview_s3_key to Parties
* Add some extra packages to Nixfile
* Add logging around custom font use
* Move app/helpers/granblue_wiki to lib/parsers/wiki
This clears up the namespace beginning with "Granblue"
* Removed some top-level Granblue libs
DataImporter and DownloadManager exist inside of the PostDeployment namespace now so these files are redundant
* Fix Downloaders namespace
Our namespace was singular Downloader, now it is plural Downloaders to match the folder name
* Fix import paths
* DownloadManager was moved to downloaders/
* import_data task now uses the PostDeployment version of DataImporter
* Update application.rb
Eager-Load/Autoload the lib/ folder
* Update cors.rb
Add Granblue website and Extension ID to CORS
* Add transformers
Transformers take raw data from Granblue Fantasy and transforms them into hensei-compatible JSON. Transformers heavily borrow from vazkii/hensei-transfer.
* Add ImportController and route
This adds the controller that handles creating a full party from transformed Granblue Fantasy data
* Adds more new items to the database
* Updates to older migrations
These were manually updated in the database
* Adds more new items to the database
* Fixes misclassification of Grand Weapons
* Add the final new items
* Delete 20250115-summons-003.csv
* Updates weapons with granblue_id reference to Characters
* Remove extra empty lines
* Adds Orologia summon
* Modify deployment manager
Use - to make it easier to copy paste into Github PR
* Fresh credentials.yml.enc
* Update .ruby-gemset
* Made PostDeploymentManager modular
We broke PostDeploymentManager out into several files to make it easier to maintain.
We also added a "force" mode that forces the script to consider all CSV files. This is useful for testing the post-deploy script itself. This should only be used in test mode or you will dirty your database.
We also fine tuned some of the logging to make sure that both verbose and non-verbose modes are helpful.
* Add table for data version and migrate
* Modify migration and re-migrate
* Create data_version.rb
Adds a model for DataVersion
* Add aws-sdk-s3 and create aws_service.rb
AwsService handles streaming game image files from the Granblue Fantasy server to our S3 instance.
* Add importers
The Importer libraries take CSV data and import them into the database for each type. We currently support characters, summons and weapons.
* Add downloaders
Downloaders take Granblue IDs and download images for those items from the Granblue Fantasy server in all relevant sizes.
Downloaders can download to disk or stream the file directly to S3.
* Create data_importer.rb
* Fetches a list of all CSV files present in the updates folder
* Checks which have already been imported
* Sends unimported data to the appropriate Importer to handle
* Create download_manager.rb
Creates an appropriate downloader for each Granblue ID it receives
* Update download_images.rake
Most of this task has been extracted into the Downloader libraries
* Update import_data.rake
* Create deploy.rake
This task is to be run as a post-deploy script. It checks for new unimported data, imports it, then downloads the relevant images to S3 or local disk depending on the parameters provided.
* Update credentials.yml.enc
* Began working on a README and added example CSVs
* Modify importer to handle updates
This way we can also add FLBs and other uncaps easier.
* Updates only require values that will change
When updating a row, fields that don't have a provided value will not be changed
* Rebuild search indices in post deploy
* Clean up logs with LoggingHelper
* More logging adjustments
Trying to get a nice-looking output
* Change some ASCII characters
* Final ASCII changes
* Fix issues with Summon and Weapon importers
* Finish README for contributing
At the expense of making database calls, the image downloader now only attempts to download images that will exist for the specific object based on how many uncaps it has.
* Remove print statements from data migration
* (Hotfix) Fix data migration failing due to nil
* Fix stale migration errors
* Ensure new characters have Awakenings
Every character starts with Balanced Lv1 awakening
* Ensure weapons without awakenings do not expose key
* Updates database seeds (#109)
* Destroy favorites when a party is deleted
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add tables used for Siero's revival (#94)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Deploy unauthenticated profile fix (#93)
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add guidebooks migration
* Implement business logic for reading Guidebooks
* Change to individual guidebook columns
* Properly output guidebook description
* Move to 1-index guidebooks
* Update party-related files for 1-index guidebooks
* Add tables for Siero
* Update download and export scripts
* Support for Siero, raids rework and edit party rework (#96)
* Add guidebooks migration
* Implement business logic for reading Guidebooks
* Change to individual guidebook columns
* Properly output guidebook description
* Move to 1-index guidebooks
* Update party-related files for 1-index guidebooks
* Add tables for Siero
* Add raid groups table
* Update raid model
To belong to the RaidGroup class
* Update job class
To have many job skills
* Add endpoint for raid groups
* Update Raid blueprint with views
* Added down for creating table
* Add guidebooks flag and auto summon flag
* Guidebooks → RaidGroup
* Auto summon → Party
* Add views to Raid blueprint
* Add views and guidebook flag to RaidGroup blueprint
* Add auto summon and Raid view to Party blueprint
* Add quick summons (#97)
* Adds quick summon migration
* Add route to update quick summon
* Add logic to update quick summon
* Remove ap call
* Add logic to updating summon uncap
You couldn't actually update summon transcendence on staging, maybe due to a bug?
Now you can. And it won't let you update outside of the bounds of what is defined on the canonical object.
* Fix summon logic
Updated logic for updating summon uncap and transcendence levels and quick summon status.
* Fix searching in Japanese (#99)
This was broken because we were using the browser-provided locale as a prefix to our method, but that is 'ja' and our methods were prefixed with 'jp'.
* Fix display of base skills (#100)
Base skills were not displaying due to a missing case in our search code. This is now fixed.
* Update awakenings (#101)
* Deploy unauthenticated profile fix (#93)
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add tables used for Siero's revival (#94)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Deploy unauthenticated profile fix (#93)
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add guidebooks migration
* Implement business logic for reading Guidebooks
* Change to individual guidebook columns
* Properly output guidebook description
* Move to 1-index guidebooks
* Update party-related files for 1-index guidebooks
* Add tables for Siero
* Adds guidebooks and gacha tables (#95)
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add tables used for Siero's revival (#94)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Deploy unauthenticated profile fix (#93)
* Implements advanced filters (#90)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix a bug where unauthenticated users couldn't view profiles (#92)
* Implements advanced filters (#90) (#91)
* Add advanced filters
Adds new filters to search:
* Full auto
* Charge attack
* Auto guard
* Number of weapons (user-selectable now)
* Number of summons
* Number of characters
* Maximum number of turns
* Maximum number of buttons
* Maximum clear time
* User quality (No anonymous users)
* Name quality (No untitled teams)
* Remixes (Only show original teams)
* Update advanced filter params
* Add default to party counter cache
* Fix being unable to see profiles when unauth
* Add guidebooks migration
* Implement business logic for reading Guidebooks
* Change to individual guidebook columns
* Properly output guidebook description
* Move to 1-index guidebooks
* Update party-related files for 1-index guidebooks
* Add tables for Siero
* Update download and export scripts
* Support for Siero, raids rework and edit party rework (#96)
* Add guidebooks migration
* Implement business logic for reading Guidebooks
* Change to individual guidebook columns
* Properly output guidebook description
* Move to 1-index guidebooks
* Update party-related files for 1-index guidebooks
* Add tables for Siero
* Add raid groups table
* Update raid model
To belong to the RaidGroup class
* Update job class
To have many job skills
* Add endpoint for raid groups
* Update Raid blueprint with views
* Added down for creating table
* Add guidebooks flag and auto summon flag
* Guidebooks → RaidGroup
* Auto summon → Party
* Add views to Raid blueprint
* Add views and guidebook flag to RaidGroup blueprint
* Add auto summon and Raid view to Party blueprint
* Remove ap call
* Add quick summons (#97)
* Adds quick summon migration
* Add route to update quick summon
* Add logic to update quick summon
* Add logic to updating summon uncap
You couldn't actually update summon transcendence on staging, maybe due to a bug?
Now you can. And it won't let you update outside of the bounds of what is defined on the canonical object.
* Fix summon logic
Updated logic for updating summon uncap and transcendence levels and quick summon status.
* Fix searching in Japanese (#99)
This was broken because we were using the browser-provided locale as a prefix to our method, but that is 'ja' and our methods were prefixed with 'jp'.
* Fix display of base skills (#100)
Base skills were not displaying due to a missing case in our search code. This is now fixed.
* Squash migrations into one file
* Add migrations
This adds migrations and data migrations for the awakening update.
* A new `Awakenings` table now exists that stores all possible awakenings, with a column for object type.
* GridCharacter and GridWeapon now have `awakening_id` and `awakening_level` fields. Their `awakening` and `awakening_type` fields were removed.
* Weapons have a mapping table, `WeaponAwakenings`, as not all weapons can be awakened.
* Data migrations are included to migrate existing user data. They should be run automatically when the migration is run with `rails db:migrate:with_data`
* Add and update models
This updates models for the awakening update.
* Awakening and WeaponAwakening models were added
* Weapon, GridWeapon and GridCharacter models get relationships to the new models defined
* GridCharacter had a validation on `awakening_level` that needed to be fixed
* Update controllers
This updates the GridCharacters and GridWeapons controllers to accept the new fields `awakening_id` and `awakening_level` from clients
* Update blueprints
This updates the blueprints to match the new schema. The new AwakeningBlueprint was accidentally added already, oops.
* Remove re-added migrations
* Implement removing job skills (#103)
This lets users remove job skills, for if they accidentally set them.
* Fix migration
