Commit graph

16 commits

Author SHA1 Message Date
cec6132823 Update wiki namespace in parsers 2025-03-02 16:19:13 -08:00
2de10d03f3 Fix path for Wiki object 2025-03-01 06:34:29 -08:00
ffbc8d0c1e Add support for passing in a Logger 2025-03-01 05:43:05 -08:00
0d997d6ad5 Add new image sizes
* Weapons can now download the “raw”image size, which is the weapon art without a background
* Characters now download the “detail” image size, which is a horizontal crop of the character’s art
* Summons now download the “detail” image size, which is a horizontal crop of the summon’s art
* Summons also download “ls” and “m” instead of “party_main” and “party_sub”, as they match the aspect ratio of weapon sizes better, which should make our lives a lot easier.
2025-03-01 05:42:40 -08:00
5955ef2650 Add support for single sizes
You can now pass one of the sizes in to only download that size for the object.
2025-03-01 05:40:34 -08:00
6db5faeb98
Add items from January/February 2025 (#184)
* Add new items

Items from:
- Unite and Fight (January 2025)
- Story Event (January 2025)
- Negima Collab (February 2025)
- January Flash Gala
- January Legend Festival
- February Flash Gala
- February character uncap

Data fixes for
- Chat Noir FLB (rarity was set to a bad value)
- End of year weapons (FLB date not set)

* Fix CSVs

* Fix data update rake task

* Add CSV require in BaseImporter
2025-02-24 16:09:30 -05:00
a042847aab
Migrate to Query Builder (#179)
* Moved queries into PartyQueryBuilder service

PartyQueryBuilder supersedes PartyQueryingConcern as it is also used for UsersController (and is our fix for profiles being broken)

* Implement PartyQueryBuilder in controllers

* Update summon_transformer.rb

This should fix the transformer so that we properly capture summons and subaura summons

* Update parties_controller_spec.rb

* Add NewRelic license key

* Add Sentry

Why not?
2025-02-12 23:43:02 -08:00
7d164b540c
Adds documentation to some lib functions (#168)
* Add sigs and docs to transformers

* Add sigs and docs to downloaders

* Adds sigs and docs to importers
2025-01-18 03:09:29 -08:00
0d5d4d5f59
Jedmund/import (#167)
* Move app/helpers/granblue_wiki to lib/parsers/wiki

This clears up the namespace beginning with "Granblue"

* Removed some top-level Granblue libs

DataImporter and DownloadManager exist inside of the PostDeployment namespace now so these files are redundant

* Fix Downloaders namespace

Our namespace was singular Downloader, now it is plural Downloaders to match the folder name

* Fix import paths

* DownloadManager was moved to downloaders/
* import_data task now uses the PostDeployment version of DataImporter

* Update application.rb

Eager-Load/Autoload the lib/ folder

* Update cors.rb

Add Granblue website and Extension ID to CORS

* Add transformers

Transformers take raw data from Granblue Fantasy and transforms them into hensei-compatible JSON. Transformers heavily borrow from vazkii/hensei-transfer.

* Add ImportController and route

This adds the controller that handles creating a full party from transformed Granblue Fantasy data
2025-01-17 12:02:12 -08:00
d75e4a8448
Downloader fixes (#162)
* Fix summon data

* Update logic for downloaders

These are responsible for downloading variants (01, 02, etc)

* Update puts statement

* Simplify ImageDownloader

* Simplify DownloadManager
2025-01-15 19:15:58 -08:00
fa386511e1
Some final post-deploy fixes (#161)
* Fix track_record error

* Add newlines

* Update PR templates
2025-01-15 18:06:20 -08:00
52f213d4cb
Update importer (#160)
* Importer now displays validation errors

This will help people debug errors before submitting their PR.

* Fix errors in the outstanding updates
2025-01-15 17:46:14 -08:00
0fd3f0f801
Importer should no longer look for recruits_id (#159) 2025-01-15 16:21:10 -08:00
d71b78e5f8
Update post-deploy script and Rails credentials (#155)
* Fresh credentials.yml.enc

* Update .ruby-gemset

* Made PostDeploymentManager modular

We broke PostDeploymentManager out into several files to make it easier to maintain.

We also added a "force" mode that forces the script to consider all CSV files. This is useful for testing the post-deploy script itself. This should only be used in test mode or you will dirty your database.

We also fine tuned some of the logging to make sure that both verbose and non-verbose modes are helpful.
2025-01-15 14:56:38 -08:00
df7454af4e
Add migration to post-deploy script (#151) 2025-01-13 06:36:01 -08:00
c0922203a7
Create pipeline for importing data via PRs (#148)
* Add table for data version and migrate

* Modify migration and re-migrate

* Create data_version.rb

Adds a model for DataVersion

* Add aws-sdk-s3 and create aws_service.rb

AwsService handles streaming game image files from the Granblue Fantasy server to our S3 instance.

* Add importers

The Importer libraries take CSV data and import them into the database for each type. We currently support characters, summons and weapons.

* Add downloaders

Downloaders take Granblue IDs and download images for those items from the Granblue Fantasy server in all relevant sizes.

Downloaders can download to disk or stream the file directly to S3.

* Create data_importer.rb

* Fetches a list of all CSV files present in the updates folder
* Checks which have already been imported
* Sends unimported data to the appropriate Importer to handle

* Create download_manager.rb

Creates an appropriate downloader for each Granblue ID it receives

* Update download_images.rake

Most of this task has been extracted into the Downloader libraries

* Update import_data.rake

* Create deploy.rake

This task is to be run as a post-deploy script. It checks for new unimported data, imports it, then downloads the relevant images to S3 or local disk depending on the parameters provided.

* Update credentials.yml.enc

* Began working on a README and added example CSVs

* Modify importer to handle updates

This way we can also add FLBs and other uncaps easier.

* Updates only require values that will change

When updating a row, fields that don't have a provided value will not be changed

* Rebuild search indices in post deploy

* Clean up logs with LoggingHelper

* More logging adjustments

Trying to get a nice-looking output

* Change some ASCII characters

* Final ASCII changes

* Fix issues with Summon and Weapon importers

* Finish README for contributing
2025-01-13 05:33:04 -08:00