Compare commits

...

29 commits

Author SHA1 Message Date
b03d5e6618
Fix error in weapons (#199) 2025-03-30 21:08:04 -07:00
309a499446
Merge pull request #198 from jedmund/jedmund/202503-legfest
Adds items from the March 2025 Legend Festival/Event
2025-03-30 21:05:39 -07:00
832bf86d47 New items from March Legfest and Scenario Event 2025-03-31 00:02:25 -04:00
65a58d8b4c
Merge pull request #197 from jedmund/jedmund/20250327-new-items
Add new items from March 2025
2025-03-26 22:44:36 -07:00
92de40bbbf Add columns for Classic II and Collab Gacha 2025-03-27 01:37:53 -04:00
aaa046c01f Add new items 2025-03-27 01:37:42 -04:00
2f04a7d3a7
Add a data miner for downloading data from GBF (#196)
* First pass at dataminer service

* Got output printing from dataminer

* Fetches summons, characters and weapons

* Can loop over objects

* Finish dataminer

Adds logger and continuing downloads
2025-03-06 19:12:44 -08:00
7880ac76cc
Add data ingestion endpoints (#195)
* Update schema.rb

* Add endpoints for importing game data

This lets privileged users import canonical data for characters, weapons and summons directly from the game
2025-03-02 17:48:35 -08:00
28a6b1894e
Delete db/migrate/20250301143956_add_wiki_raw_to_characters.rb (#194) 2025-03-02 16:29:05 -08:00
3746ee9af6
Merge pull request #193 from jedmund/jedmund/wiki-fields
Implement columns for raw wiki/game data
2025-03-02 16:20:55 -08:00
cec6132823 Update wiki namespace in parsers 2025-03-02 16:19:13 -08:00
311c218863 Adds a task for fetching data from wiki
Can be used to fetch one object or multiple
2025-03-02 16:17:32 -08:00
c060a4525b Update migration
Now this adds raw data columns to weapons, summons, and characters
2025-03-02 16:08:35 -08:00
2b8dfe9e20 Merge branch 'main' into jedmund/wiki-fields 2025-03-01 07:32:17 -08:00
b1800f411f
Cleans up data on some Characters (#192)
This should update the `wiki_en` field for Kaguya, Fenie, and Uriel. It also adds `character_id` for 6 characters that were missing a value.
2025-03-01 07:31:49 -08:00
2de10d03f3 Fix path for Wiki object 2025-03-01 06:34:29 -08:00
a6ede6ecf7
Merge pull request #191 from jedmund/jedmund/parallelize-downloaders
Parallelize downloaders and update sizes
2025-03-01 05:46:08 -08:00
e75578bea3 Refactor download_all_images task
This refactor focuses on implementing parallelization. This allows us to pass in a number of threads and download concurrently. This makes downloading lots of images a lot faster.
2025-03-01 05:43:57 -08:00
ffbc8d0c1e Add support for passing in a Logger 2025-03-01 05:43:05 -08:00
0d997d6ad5 Add new image sizes
* Weapons can now download the “raw”image size, which is the weapon art without a background
* Characters now download the “detail” image size, which is a horizontal crop of the character’s art
* Summons now download the “detail” image size, which is a horizontal crop of the summon’s art
* Summons also download “ls” and “m” instead of “party_main” and “party_sub”, as they match the aspect ratio of weapon sizes better, which should make our lives a lot easier.
2025-03-01 05:42:40 -08:00
5955ef2650 Add support for single sizes
You can now pass one of the sizes in to only download that size for the object.
2025-03-01 05:40:34 -08:00
ae62d594a8 Add parallel gem 2025-03-01 05:29:21 -08:00
82b3d0ed88
Merge pull request #190 from jedmund/jedmund/add-recruits
Adds recruits value to new weapons
2025-03-01 05:25:15 -08:00
e1d983a6d4 Adds recruits IDs to new weapons 2025-02-28 18:24:15 -08:00
4d3c1a800b
Update config files (#189)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Update .gitignore

Hide backups and logs directories, since we’ll be storing these in the project folder. Also hide mise’s .local directory.

* Change NewRelic log directory

Moved from log/ to logs/

* Add rake task for backing up/restoring prod db

* Rubocop fixes

* Fix error where :preview_state didn’t have an attribute

* Add supervisord ini

This uses my local paths, so we should try to abstract that away later.

* Ignore mise.toml
2025-02-27 23:13:57 -08:00
b2d2952b35
Fix limit column in weapon migration (#188)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Add items from late 2025-02

* Yuel (Grand)
* Tsukuyomi
* Sennen Goji
* Nightgaze Gale
* Bane of Avidya
* Klesha-Cleaning Dharmachakra

* Add default value for limit column
2025-02-27 20:41:51 -08:00
6bcbc97566
Jedmund/202502 update (#187)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Add items from late 2025-02

* Yuel (Grand)
* Tsukuyomi
* Sennen Goji
* Nightgaze Gale
* Bane of Avidya
* Klesha-Cleaning Dharmachakra
2025-02-27 20:27:43 -08:00
9827658771
Render the correct view when updating jobs (#186)
We were referencing an old view in `PartiesController#update_job`. This has been corrected.
2025-02-25 09:11:13 -05:00
505176ae5f
Update api URL (#185)
This change updates the production API URL to https://api.granblue.team/v1 instead of https://api.granblue.team/api/v1.
2025-02-25 08:23:47 -05:00
36 changed files with 879 additions and 205 deletions

6
.gitignore vendored
View file

@ -37,9 +37,12 @@
# Ignore master key for decrypting credentials and more. # Ignore master key for decrypting credentials and more.
/config/master.key /config/master.key
# Ignore exported and downloaded files # Ignore specific directories
/.local
/export /export
/download /download
/backups
/logs
.DS_Store .DS_Store
@ -55,3 +58,4 @@ config/application.yml
# Ignore AI Codebase-generated files # Ignore AI Codebase-generated files
codebase.md codebase.md
mise.toml

View file

@ -76,10 +76,13 @@ gem 'strscan'
# New Relic Ruby Agent # New Relic Ruby Agent
gem 'newrelic_rpm' gem 'newrelic_rpm'
# Parallel processing made simple and fast
gem 'parallel'
# The Sentry SDK for Rails # The Sentry SDK for Rails
gem "stackprof" gem 'sentry-rails'
gem "sentry-ruby" gem 'sentry-ruby'
gem "sentry-rails" gem 'stackprof'
group :doc do group :doc do
gem 'apipie-rails' gem 'apipie-rails'

View file

@ -482,6 +482,7 @@ DEPENDENCIES
mini_magick mini_magick
newrelic_rpm newrelic_rpm
oj oj
parallel
pg pg
pg_query pg_query
pg_search pg_search

View file

@ -131,6 +131,7 @@ module Api
fields :edit_key fields :edit_key
end end
# Remixed view
view :remixed do view :remixed do
include_view :created include_view :created
include_view :source_party include_view :source_party

View file

@ -27,6 +27,8 @@ module Api
6 => 5 6 => 5
}.freeze }.freeze
before_action :ensure_admin_role, only: %i[weapons summons characters]
## ##
# Processes an import request. # Processes an import request.
# #
@ -51,7 +53,7 @@ module Api
unless raw_params['deck'].is_a?(Hash) && unless raw_params['deck'].is_a?(Hash) &&
raw_params['deck'].key?('pc') && raw_params['deck'].key?('pc') &&
raw_params['deck'].key?('npc') raw_params['deck'].key?('npc')
Rails.logger.error "[IMPORT] Deck data incomplete or missing." Rails.logger.error '[IMPORT] Deck data incomplete or missing.'
return render json: { error: 'Invalid deck data' }, status: :unprocessable_content return render json: { error: 'Invalid deck data' }, status: :unprocessable_content
end end
@ -68,8 +70,111 @@ module Api
render json: { error: e.message }, status: :unprocessable_content render json: { error: e.message }, status: :unprocessable_content
end end
def weapons
Rails.logger.info '[IMPORT] Checking weapon gamedata input...'
body = parse_request_body
return unless body
weapon = Weapon.find_by(granblue_id: body['id'])
unless weapon
Rails.logger.error "[IMPORT] Weapon not found with ID: #{body['id']}"
return render json: { error: 'Weapon not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
weapon.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Weapon gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update weapon gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
def summons
Rails.logger.info '[IMPORT] Checking summon gamedata input...'
body = parse_request_body
return unless body
summon = Summon.find_by(granblue_id: body['id'])
unless summon
Rails.logger.error "[IMPORT] Summon not found with ID: #{body['id']}"
return render json: { error: 'Summon not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
summon.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Summon gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update summon gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
##
# Updates character gamedata from JSON blob.
#
# @return [void] Renders JSON response with success or error message.
def characters
Rails.logger.info '[IMPORT] Checking character gamedata input...'
body = parse_request_body
return unless body
character = Character.find_by(granblue_id: body['id'])
unless character
Rails.logger.error "[IMPORT] Character not found with ID: #{body['id']}"
return render json: { error: 'Character not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
character.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Character gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update character gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
private private
##
# Ensures the current user has admin role (role 9).
# Renders an error if the user is not an admin.
#
# @return [void]
def ensure_admin_role
return if current_user&.role == 9
Rails.logger.error "[IMPORT] Unauthorized access attempt by user #{current_user&.id}"
render json: { error: 'Unauthorized' }, status: :unauthorized
end
## ##
# Reads and parses the raw JSON request body. # Reads and parses the raw JSON request body.
# #

View file

@ -47,7 +47,7 @@ module Api
end end
end end
render json: PartyBlueprint.render(@party, view: :jobs) if @party.save! render json: PartyBlueprint.render(@party, view: :job_metadata) if @party.save!
end end
def update_job_skills def update_job_skills

View file

@ -78,6 +78,7 @@ class Party < ApplicationRecord
include GranblueEnums include GranblueEnums
# Define preview_state as an enum. # Define preview_state as an enum.
attribute :preview_state, :integer
enum :preview_state, { pending: 0, queued: 1, in_progress: 2, generated: 3, failed: 4 } enum :preview_state, { pending: 0, queued: 1, in_progress: 2, generated: 3, failed: 4 }
# ActiveRecord Associations # ActiveRecord Associations
@ -400,7 +401,7 @@ class Party < ApplicationRecord
# @return [void] # @return [void]
def skills_are_unique def skills_are_unique
validate_uniqueness_of_associations([skill0, skill1, skill2, skill3], validate_uniqueness_of_associations([skill0, skill1, skill2, skill3],
[:skill0, :skill1, :skill2, :skill3], %i[skill0 skill1 skill2 skill3],
:job_skills) :job_skills)
end end
@ -410,7 +411,7 @@ class Party < ApplicationRecord
# @return [void] # @return [void]
def guidebooks_are_unique def guidebooks_are_unique
validate_uniqueness_of_associations([guidebook1, guidebook2, guidebook3], validate_uniqueness_of_associations([guidebook1, guidebook2, guidebook3],
[:guidebook1, :guidebook2, :guidebook3], %i[guidebook1 guidebook2 guidebook3],
:guidebooks) :guidebooks)
end end
@ -438,7 +439,7 @@ class Party < ApplicationRecord
def update_element! def update_element!
main_weapon = weapons.detect { |gw| gw.position.to_i == -1 } main_weapon = weapons.detect { |gw| gw.position.to_i == -1 }
new_element = main_weapon&.weapon&.element new_element = main_weapon&.weapon&.element
update_column(:element, new_element) if new_element.present? && self.element != new_element update_column(:element, new_element) if new_element.present? && element != new_element
end end
## ##
@ -449,7 +450,7 @@ class Party < ApplicationRecord
# @return [void] # @return [void]
def update_extra! def update_extra!
new_extra = weapons.any? { |gw| GridWeapon::EXTRA_POSITIONS.include?(gw.position.to_i) } new_extra = weapons.any? { |gw| GridWeapon::EXTRA_POSITIONS.include?(gw.position.to_i) }
update_column(:extra, new_extra) if self.extra != new_extra update_column(:extra, new_extra) if extra != new_extra
end end
## ##

251
app/services/dataminer.rb Normal file
View file

@ -0,0 +1,251 @@
# frozen_string_literal: true
class Dataminer
include HTTParty
BOT_UID = '39094985'
GAME_VERSION = '1741068713'
base_uri 'https://game.granbluefantasy.jp'
format :json
HEADERS = {
'Accept' => 'application/json, text/javascript, */*; q=0.01',
'Accept-Language' => 'en-US,en;q=0.9',
'Accept-Encoding' => 'gzip, deflate, br, zstd',
'Content-Type' => 'application/json',
'DNT' => '1',
'Origin' => 'https://game.granbluefantasy.jp',
'Referer' => 'https://game.granbluefantasy.jp/',
'User-Agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36',
'X-Requested-With' => 'XMLHttpRequest'
}.freeze
attr_reader :page, :cookies, :logger, :debug
def initialize(page:, access_token:, wing:, midship:, t: 'dummy', debug: false)
@page = page
@cookies = {
access_gbtk: access_token,
wing: wing,
t: t,
midship: midship
}
@debug = debug
setup_logger
end
def fetch
timestamp = Time.now.to_i * 1000
response = self.class.post(
"/#{page}?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}",
headers: HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
)
)
raise AuthenticationError if auth_failed?(response)
response
end
def fetch_character(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/npc_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
style: 1,
character_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Character', granblue_id, response) if response
response
end
def fetch_weapon(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/weapon_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
weapon_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Weapon', granblue_id, response) if response
response
end
def fetch_summon(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/summon_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
summon_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Summon', granblue_id, response) if response
response
end
# Public batch processing methods
def fetch_all_characters(only_missing: false)
process_all_records('Character', only_missing: only_missing)
end
def fetch_all_weapons(only_missing: false)
process_all_records('Weapon', only_missing: only_missing)
end
def fetch_all_summons(only_missing: false)
process_all_records('Summon', only_missing: only_missing)
end
private
def format_cookies
cookies.map { |k, v| "#{k}=#{v}" }.join('; ')
end
def auth_failed?(response)
return true if response.code != 200
begin
parsed = JSON.parse(response.body)
parsed.is_a?(Hash) && parsed['auth_status'] == 'require_auth'
rescue JSON::ParserError
true
end
end
def setup_logger
@logger = ::Logger.new($stdout)
@logger.level = debug ? ::Logger::DEBUG : ::Logger::INFO
@logger.formatter = proc do |severity, _datetime, _progname, msg|
case severity
when 'DEBUG'
debug ? "#{msg}\n" : ''
else
"#{msg}\n"
end
end
# Suppress SQL logs in non-debug mode
return if debug
ActiveRecord::Base.logger.level = ::Logger::INFO if defined?(ActiveRecord::Base)
end
def fetch_detail(url, body)
logger.debug "\n=== Request Details ==="
logger.debug "URL: #{url}"
logger.debug 'Headers:'
logger.debug HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
).inspect
logger.debug 'Body:'
logger.debug body.to_json
logger.debug '===================='
response = self.class.post(
url,
headers: HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
),
body: body.to_json
)
logger.debug "\n=== Response Details ==="
logger.debug "Response code: #{response.code}"
logger.debug 'Response headers:'
logger.debug response.headers.inspect
logger.debug 'Raw response body:'
logger.debug response.body.inspect
begin
logger.debug 'Parsed response body (if JSON):'
logger.debug JSON.parse(response.body).inspect
rescue JSON::ParserError => e
logger.debug "Could not parse as JSON: #{e.message}"
end
logger.debug '======================'
raise AuthenticationError if auth_failed?(response)
JSON.parse(response.body)
end
def update_game_data(model_name, granblue_id, response_data)
return unless response_data.is_a?(Hash)
model = Object.const_get(model_name)
record = model.find_by(granblue_id: granblue_id)
if record
record.update(game_raw_en: response_data)
logger.debug "Updated #{model_name} #{granblue_id}"
else
logger.warn "#{model_name} with granblue_id #{granblue_id} not found in database"
end
rescue StandardError => e
logger.error "Error updating #{model_name} #{granblue_id}: #{e.message}"
end
def process_all_records(model_name, only_missing: false)
model = Object.const_get(model_name)
scope = model
scope = scope.where(game_raw_en: nil) if only_missing
total = scope.count
success_count = 0
error_count = 0
logger.info "Starting to fetch #{total} #{model_name.downcase}s#{' (missing data only)' if only_missing}..."
scope.find_each do |record|
logger.info "\nProcessing #{model_name} #{record.granblue_id} (#{success_count + error_count + 1}/#{total})"
response = case model_name
when 'Character'
fetch_character(record.granblue_id)
when 'Weapon'
fetch_weapon(record.granblue_id)
when 'Summon'
fetch_summon(record.granblue_id)
end
success_count += 1
logger.debug "Successfully processed #{model_name} #{record.granblue_id}"
sleep(1)
rescue StandardError => e
error_count += 1
logger.error "Error processing #{model_name} #{record.granblue_id}: #{e.message}"
end
logger.info "\nProcessing complete!"
logger.info "Total: #{total}"
logger.info "Successful: #{success_count}"
logger.info "Failed: #{error_count}"
end
class AuthenticationError < StandardError; end
end

View file

@ -25,6 +25,8 @@ common: &default_settings
# agent_enabled: false # agent_enabled: false
log_file_path: logs/
# Logging level for log/newrelic_agent.log # Logging level for log/newrelic_agent.log
log_level: info log_level: info

View file

@ -4,20 +4,20 @@
# the maximum value specified for Puma. Default is set to 5 threads for minimum # the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record. # and maximum; this matches the default thread size of Active Record.
# #
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 } max_threads_count = ENV.fetch('RAILS_MAX_THREADS') { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count } min_threads_count = ENV.fetch('RAILS_MIN_THREADS') { max_threads_count }
threads min_threads_count, max_threads_count threads min_threads_count, max_threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000. # Specifies the `port` that Puma will listen on to receive requests; default is 3000.
# #
port ENV.fetch("PORT") { 3000 } port ENV.fetch('PORT', 3000)
# Specifies the `environment` that Puma will run in. # Specifies the `environment` that Puma will run in.
# #
environment ENV.fetch("RAILS_ENV") { "development" } environment ENV.fetch('RAILS_ENV') { 'development' }
# Specifies the `pidfile` that Puma will use. # Specifies the `pidfile` that Puma will use.
pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" } pidfile ENV.fetch('PIDFILE') { 'tmp/pids/server.pid' }
# Specifies the number of `workers` to boot in clustered mode. # Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together # Workers are forked web server processes. If using threads and workers together

View file

@ -4,8 +4,9 @@ Rails.application.routes.draw do
skip_controllers :applications, :authorized_applications skip_controllers :applications, :authorized_applications
end end
namespace :api, defaults: { format: :json } do path_prefix = Rails.env.production? ? '/v1' : '/api/v1'
namespace :v1 do
scope path: path_prefix, module: 'api/v1', defaults: { format: :json } do
resources :parties, only: %i[index create update destroy] resources :parties, only: %i[index create update destroy]
resources :users, only: %i[create update show] resources :users, only: %i[create update show]
resources :grid_weapons, only: %i[update destroy] resources :grid_weapons, only: %i[update destroy]
@ -19,6 +20,9 @@ Rails.application.routes.draw do
get 'version', to: 'api#version' get 'version', to: 'api#version'
post 'import', to: 'import#create' post 'import', to: 'import#create'
post 'import/weapons', to: 'import#weapons'
post 'import/summons', to: 'import#summons'
post 'import/characters', to: 'import#characters'
get 'users/info/:id', to: 'users#info' get 'users/info/:id', to: 'users#info'
@ -74,7 +78,6 @@ Rails.application.routes.draw do
delete 'favorites', to: 'favorites#destroy' delete 'favorites', to: 'favorites#destroy'
end end
end
if Rails.env.development? if Rails.env.development?
get '/party-previews/*filename', to: proc { |env| get '/party-previews/*filename', to: proc { |env|

View file

@ -6,119 +6,119 @@ class MigrateWeaponSeries < ActiveRecord::Migration[8.0]
puts 'Starting weapon series migration...' puts 'Starting weapon series migration...'
puts 'Updating Seraphic Weapons (0 -> 1)...' puts 'Updating Seraphic Weapons (0 -> 1)...'
Weapon.find_by(series: 0).update!(new_series: 1) Weapon.where(series: 0).update_all(new_series: 1)
puts 'Updating Grand Weapons (1 -> 2)...' puts 'Updating Grand Weapons (1 -> 2)...'
Weapon.find_by(series: 1).update!(new_series: 2) Weapon.where(series: 1).update_all(new_series: 2)
puts 'Updating Dark Opus Weapons (2 -> 3)...' puts 'Updating Dark Opus Weapons (2 -> 3)...'
Weapon.find_by(series: 2).update!(new_series: 3) Weapon.where(series: 2).update_all(new_series: 3)
puts 'Updating Revenant Weapons (4 -> 4)...' puts 'Updating Revenant Weapons (4 -> 4)...'
Weapon.find_by(series: 4).update!(new_series: 4) Weapon.where(series: 4).update_all(new_series: 4)
puts 'Updating Primal Weapons (6 -> 5)...' puts 'Updating Primal Weapons (6 -> 5)...'
Weapon.find_by(series: 6).update!(new_series: 5) Weapon.where(series: 6).update_all(new_series: 5)
puts 'Updating Beast Weapons (5, 7 -> 6)...' puts 'Updating Beast Weapons (5, 7 -> 6)...'
Weapon.find_by(series: 5).update!(new_series: 6) Weapon.where(series: 5).update_all(new_series: 6)
Weapon.find_by(series: 7).update!(new_series: 6) Weapon.where(series: 7).update_all(new_series: 6)
puts 'Updating Regalia Weapons (8 -> 7)...' puts 'Updating Regalia Weapons (8 -> 7)...'
Weapon.find_by(series: 8).update!(new_series: 7) Weapon.where(series: 8).update_all(new_series: 7)
puts 'Updating Omega Weapons (9 -> 8)...' puts 'Updating Omega Weapons (9 -> 8)...'
Weapon.find_by(series: 9).update!(new_series: 8) Weapon.where(series: 9).update_all(new_series: 8)
puts 'Updating Olden Primal Weapons (10 -> 9)...' puts 'Updating Olden Primal Weapons (10 -> 9)...'
Weapon.find_by(series: 10).update!(new_series: 9) Weapon.where(series: 10).update_all(new_series: 9)
puts 'Updating Hollowsky Weapons (12 -> 10)...' puts 'Updating Hollowsky Weapons (12 -> 10)...'
Weapon.find_by(series: 12).update!(new_series: 10) Weapon.where(series: 12).update_all(new_series: 10)
puts 'Updating Xeno Weapons (13 -> 11)...' puts 'Updating Xeno Weapons (13 -> 11)...'
Weapon.find_by(series: 13).update!(new_series: 11) Weapon.where(series: 13).update_all(new_series: 11)
puts 'Updating Rose Weapons (15 -> 12)...' puts 'Updating Rose Weapons (15 -> 12)...'
Weapon.find_by(series: 15).update!(new_series: 12) Weapon.where(series: 15).update_all(new_series: 12)
puts 'Updating Ultima Weapons (17 -> 13)...' puts 'Updating Ultima Weapons (17 -> 13)...'
Weapon.find_by(series: 17).update!(new_series: 13) Weapon.where(series: 17).update_all(new_series: 13)
puts 'Updating Bahamut Weapons (16 -> 14)...' puts 'Updating Bahamut Weapons (16 -> 14)...'
Weapon.find_by(series: 16).update!(new_series: 14) Weapon.where(series: 16).update_all(new_series: 14)
puts 'Updating Epic Weapons (18 -> 15)...' puts 'Updating Epic Weapons (18 -> 15)...'
Weapon.find_by(series: 18).update!(new_series: 15) Weapon.where(series: 18).update_all(new_series: 15)
puts 'Updating Cosmos Weapons (20 -> 16)...' puts 'Updating Cosmos Weapons (20 -> 16)...'
Weapon.find_by(series: 20).update!(new_series: 16) Weapon.where(series: 20).update_all(new_series: 16)
puts 'Updating Superlative Weapons (22 -> 17)...' puts 'Updating Superlative Weapons (22 -> 17)...'
Weapon.find_by(series: 22).update!(new_series: 17) Weapon.where(series: 22).update_all(new_series: 17)
puts 'Updating Vintage Weapons (23 -> 18)...' puts 'Updating Vintage Weapons (23 -> 18)...'
Weapon.find_by(series: 23).update!(new_series: 18) Weapon.where(series: 23).update_all(new_series: 18)
puts 'Updating Class Champion Weapons (24 -> 19)...' puts 'Updating Class Champion Weapons (24 -> 19)...'
Weapon.find_by(series: 24).update!(new_series: 19) Weapon.where(series: 24).update_all(new_series: 19)
puts 'Updating Sephira Weapons (28 -> 23)...' puts 'Updating Sephira Weapons (28 -> 23)...'
Weapon.find_by(series: 28).update!(new_series: 23) Weapon.where(series: 28).update_all(new_series: 23)
puts 'Updating Astral Weapons (14 -> 26)...' puts 'Updating Astral Weapons (14 -> 26)...'
Weapon.find_by(series: 14).update!(new_series: 26) Weapon.where(series: 14).update_all(new_series: 26)
puts 'Updating Draconic Weapons (3 -> 27)...' puts 'Updating Draconic Weapons (3 -> 27)...'
Weapon.find_by(series: 3).update!(new_series: 27) Weapon.where(series: 3).update_all(new_series: 27)
puts 'Updating Ancestral Weapons (21 -> 29)...' puts 'Updating Ancestral Weapons (21 -> 29)...'
Weapon.find_by(series: 21).update!(new_series: 29) Weapon.where(series: 21).update_all(new_series: 29)
puts 'Updating New World Foundation (29 -> 30)...' puts 'Updating New World Foundation (29 -> 30)...'
Weapon.find_by(series: 29).update!(new_series: 30) Weapon.where(series: 29).update_all(new_series: 30)
puts 'Updating Ennead Weapons (19 -> 31)...' puts 'Updating Ennead Weapons (19 -> 31)...'
Weapon.find_by(series: 19).update!(new_series: 31) Weapon.where(series: 19).update_all(new_series: 31)
puts 'Updating Militis Weapons (11 -> 32)...' puts 'Updating Militis Weapons (11 -> 32)...'
Weapon.find_by(series: 11).update!(new_series: 32) Weapon.where(series: 11).update_all(new_series: 32)
puts 'Updating Malice Weapons (26 -> 33)...' puts 'Updating Malice Weapons (26 -> 33)...'
Weapon.find_by(series: 26).update!(new_series: 33) Weapon.where(series: 26).update_all(new_series: 33)
puts 'Updating Menace Weapons (26 -> 34)...' puts 'Updating Menace Weapons (26 -> 34)...'
Weapon.find_by(series: 26).update!(new_series: 34) Weapon.where(series: 26).update_all(new_series: 34)
puts 'Updating Illustrious Weapons (31 -> 35)...' puts 'Updating Illustrious Weapons (31 -> 35)...'
Weapon.find_by(series: 31).update!(new_series: 35) Weapon.where(series: 31).update_all(new_series: 35)
puts 'Updating Proven Weapons (25 -> 36)...' puts 'Updating Proven Weapons (25 -> 36)...'
Weapon.find_by(series: 25).update!(new_series: 36) Weapon.where(series: 25).update_all(new_series: 36)
puts 'Updating Revans Weapons (30 -> 37)...' puts 'Updating Revans Weapons (30 -> 37)...'
Weapon.find_by(series: 30).update!(new_series: 37) Weapon.where(series: 30).update_all(new_series: 37)
puts 'Updating World Weapons (32 -> 38)...' puts 'Updating World Weapons (32 -> 38)...'
Weapon.find_by(series: 32).update!(new_series: 38) Weapon.where(series: 32).update_all(new_series: 38)
puts 'Updating Exo Weapons (33 -> 39)...' puts 'Updating Exo Weapons (33 -> 39)...'
Weapon.find_by(series: 33).update!(new_series: 39) Weapon.where(series: 33).update_all(new_series: 39)
puts 'Updating Draconic Weapons Providence (34 -> 40)...' puts 'Updating Draconic Weapons Providence (34 -> 40)...'
Weapon.find_by(series: 34).update!(new_series: 40) Weapon.where(series: 34).update_all(new_series: 40)
puts 'Updating Celestial Weapons (37 -> 41)...' puts 'Updating Celestial Weapons (37 -> 41)...'
Weapon.find_by(series: 37).update!(new_series: 41) Weapon.where(series: 37).update_all(new_series: 41)
puts 'Updating Omega Rebirth Weapons (38 -> 42)...' puts 'Updating Omega Rebirth Weapons (38 -> 42)...'
Weapon.find_by(series: 38).update!(new_series: 42) Weapon.where(series: 38).update_all(new_series: 42)
puts 'Updating Event Weapons (34 -> 98)...' puts 'Updating Event Weapons (34 -> 98)...'
Weapon.find_by(series: 34).update!(new_series: 98) # Event Weapon.where(series: 34).update_all(new_series: 98) # Event
puts 'Updating Gacha Weapons (36 -> 99)...' puts 'Updating Gacha Weapons (36 -> 99)...'
Weapon.find_by(series: 36).update!(new_series: 99) # Gacha Weapon.where(series: 36).update_all(new_series: 99) # Gacha
puts 'Migration completed successfully!' puts 'Migration completed successfully!'
rescue StandardError => e rescue StandardError => e

View file

@ -0,0 +1,15 @@
class AddRawDataColumns < ActiveRecord::Migration[8.0]
def change
add_column :characters, :wiki_raw, :text
add_column :characters, :game_raw_en, :text
add_column :characters, :game_raw_jp, :text
add_column :summons, :wiki_raw, :text
add_column :summons, :game_raw_en, :text
add_column :summons, :game_raw_jp, :text
add_column :weapons, :wiki_raw, :text
add_column :weapons, :game_raw_en, :text
add_column :weapons, :game_raw_jp, :text
end
end

View file

@ -0,0 +1,6 @@
class AddClassicIiAndCollabToGacha < ActiveRecord::Migration[8.0]
def change
add_column :gacha, :classic_ii, :boolean, default: false
add_column :gacha, :collab, :boolean, default: false
end
end

View file

@ -10,7 +10,7 @@
# #
# It's strongly recommended that you check this file into your version control system. # It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do ActiveRecord::Schema[8.0].define(version: 2025_03_01_143956) do
# These are extensions that must be enabled in order to support this database # These are extensions that must be enabled in order to support this database
enable_extension "btree_gin" enable_extension "btree_gin"
enable_extension "pg_catalog.plpgsql" enable_extension "pg_catalog.plpgsql"
@ -67,6 +67,9 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.string "kamigame", default: "" t.string "kamigame", default: ""
t.string "nicknames_en", default: [], null: false, array: true t.string "nicknames_en", default: [], null: false, array: true
t.string "nicknames_jp", default: [], null: false, array: true t.string "nicknames_jp", default: [], null: false, array: true
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_characters_on_granblue_id" t.index ["granblue_id"], name: "index_characters_on_granblue_id"
t.index ["name_en"], name: "index_characters_on_name_en", opclass: :gin_trgm_ops, using: :gin t.index ["name_en"], name: "index_characters_on_name_en", opclass: :gin_trgm_ops, using: :gin
end end
@ -420,6 +423,9 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.date "transcendence_date" t.date "transcendence_date"
t.string "nicknames_en", default: [], null: false, array: true t.string "nicknames_en", default: [], null: false, array: true
t.string "nicknames_jp", default: [], null: false, array: true t.string "nicknames_jp", default: [], null: false, array: true
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_summons_on_granblue_id" t.index ["granblue_id"], name: "index_summons_on_granblue_id"
t.index ["name_en"], name: "index_summons_on_name_en", opclass: :gin_trgm_ops, using: :gin t.index ["name_en"], name: "index_summons_on_name_en", opclass: :gin_trgm_ops, using: :gin
end end
@ -495,6 +501,10 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.date "transcendence_date" t.date "transcendence_date"
t.string "recruits" t.string "recruits"
t.integer "series" t.integer "series"
t.integer "new_series"
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_weapons_on_granblue_id" t.index ["granblue_id"], name: "index_weapons_on_granblue_id"
t.index ["name_en"], name: "index_weapons_on_name_en", opclass: :gin_trgm_ops, using: :gin t.index ["name_en"], name: "index_weapons_on_name_en", opclass: :gin_trgm_ops, using: :gin
t.index ["recruits"], name: "index_weapons_on_recruits" t.index ["recruits"], name: "index_weapons_on_recruits"

View file

@ -0,0 +1,3 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Yuel (Grand),ユエル(リミテッドver),3040580000,3,2,10,8,2,2,,false,265,1375,,1413,7335,,,,,,false,false,,,{3006},Yuel (Grand),2025-02-27,,,ユエル (SSR)リミテッドバージョン,486749,SSRリミテッドユエル,,
Tsukuyomi,ツクヨミ,3040581000,3,5,6,,2,5,,false,335,1175,,1785,6275,,,,,,false,false,,,{3270},Tsukuyomi,2025-02-27,,,ツクヨミ (SSR),486748,SSRツクヨミ,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Yuel (Grand) ユエル(リミテッドver) 3040580000 3 2 10 8 2 2 false 265 1375 1413 7335 false false {3006} Yuel (Grand) 2025-02-27 ユエル (SSR)リミテッドバージョン 486749 SSRリミテッドユエル
3 Tsukuyomi ツクヨミ 3040581000 3 5 6 2 5 false 335 1175 1785 6275 false false {3270} Tsukuyomi 2025-02-27 ツクヨミ (SSR) 486748 SSRツクヨミ

View file

@ -0,0 +1,7 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
Sennen Goji,千年護持,1040917100,3,2,10,1,true,false,150,15,29,244,,,519,3577,,,false,,false,false,,,2025-02-28,2025-02-28,,Sennen Goji,,486774,千年護持,,,false,,,,,,,,,,,,,,
Nightgaze Gate,夜見之門,1040424200,3,5,6,99,false,false,100,10,46,285,,,369,2163,,,false,,false,false,,,2025-02-28,,,Nightgaze Gate,,,夜見之門,,,false,,,,,,,,,,,,,,
Bane of Avidya,無明滅却杵,1040424100,3,6,6,98,false,false,100,10,34,293,,,227,1744,,,false,,false,false,,,2025-02-26,,,Bane of Avidya,無明滅却杵,486167,無明滅却杵,,,false,,,,,,,,,,,,,,
"
Klesha-Cleansing Dharmachakra",破魔輪宝,1030609800,2,6,7,98,false,false,75,10,22,136,,,197,1272,,,false,,false,false,,,2025-02-26,,,"
Klesha-Cleansing Dharmachakra",破魔輪宝,,破魔輪宝,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 Sennen Goji 千年護持 1040917100 3 2 10 1 true false 150 15 29 244 519 3577 false false false 2025-02-28 2025-02-28 Sennen Goji 486774 千年護持 false
3 Nightgaze Gate 夜見之門 1040424200 3 5 6 99 false false 100 10 46 285 369 2163 false false false 2025-02-28 Nightgaze Gate 夜見之門 false
4 Bane of Avidya 無明滅却杵 1040424100 3 6 6 98 false false 100 10 34 293 227 1744 false false false 2025-02-26 Bane of Avidya 無明滅却杵 486167 無明滅却杵 false
5 Klesha-Cleansing Dharmachakra 破魔輪宝 1030609800 2 6 7 98 false false 75 10 22 136 197 1272 false false false 2025-02-26 Klesha-Cleansing Dharmachakra 破魔輪宝 破魔輪宝 false

View file

@ -0,0 +1,9 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
,,1040619800,,,,,,,,,,,,,,,,,,,,,3040572000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040713900,,,,,,,,,,,,,,,,,,,,,3040573000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040028200,,,,,,,,,,,,,,,,,,,,,3040577000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040121200,,,,,,,,,,,,,,,,,,,,,3040576000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040219700,,,,,,,,,,,,,,,,,,,,,3040579000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040319200,,,,,,,,,,,,,,,,,,,,,3040578000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040917100,,,,,,,,,,,,,,,,,,,,,3040580000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040424200,,,,,,,,,,,,,,,,,,,,,3040581000,,,,,,,,,,,,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 1040619800 3040572000
3 1040713900 3040573000
4 1040028200 3040577000
5 1040121200 3040576000
6 1040219700 3040579000
7 1040319200 3040578000
8 1040917100 3040580000
9 1040424200 3040581000

View file

@ -0,0 +1,10 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Kaguya (Grand),,3040486000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Kaguya,
,,3040519000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Fenie,
,,3040501000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Uriel,
,,3040254000,,,,,,,,,,,,,,,,,,,,,,,{2030},,,,,,,,,
,,3040158000,,,,,,,,,,,,,,,,,,,,,,,{3098},,,,,,,,,
,,3040103000,,,,,,,,,,,,,,,,,,,,,,,{3070},,,,,,,,,
,,3040377000,,,,,,,,,,,,,,,,,,,,,,,{3192},,,,,,,,,
,,3040058000,,,,,,,,,,,,,,,,,,,,,,,{3045},,,,,,,,,
,,3040500000,,,,,,,,,,,,,,,,,,,,,,,{1027},,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Kaguya (Grand) 3040486000 Kaguya
3 3040519000 Fenie
4 3040501000 Uriel
5 3040254000 {2030}
6 3040158000 {3098}
7 3040103000 {3070}
8 3040377000 {3192}
9 3040058000 {3045}
10 3040500000 {1027}

View file

@ -0,0 +1,7 @@
name_en,name_jp,granblue_id,rarity,element,series,flb,ulb,max_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,subaura,limit,transcendence,max_atk_xlb,max_hp_xlb,summon_id,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,transcendence_date,nicknames_en,nicknames_jp
,,2040361000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040363000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040368000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040366000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040381000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040385000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
1 name_en name_jp granblue_id rarity element series flb ulb max_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb subaura limit transcendence max_atk_xlb max_hp_xlb summon_id release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame transcendence_date nicknames_en nicknames_jp
2 2040361000 true 2025-03-10
3 2040363000 true 2025-03-10
4 2040368000 true 2025-03-10
5 2040366000 true 2025-03-10
6 2040381000 true 2025-03-10
7 2040385000 true 2025-03-10

View file

@ -0,0 +1,5 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Basara (Grand),バサラ (リミテッドver),3040582000,3,6,10,7,1,2,,false,192,1128,,1740,8760,,,,,,false,false,,,{3271},Basara,2025-03-17,,,バサラ (SSR)リミテッドバージョン,489323,SSRバサラ,,
Mahira (Summer),マキラ(水着ver),3040584000,3,1,8,7,2,3,,false,281,1217,,1408,8915,,,,,,false,false,,,{3073},Mahira (Summer),2025-03-17,,,マキラ (SSR)水着バージョン,489328,SSR水着マキラ,,
Lu Woh (Summer),ル・オー(水着ver),3040583000,3,4,7,6,0,2,,false,236,1172,,1385,7820,,,,,,false,false,,,{3221},Lu Woh (Summer),2025-03-17,,,ル・オー (SSR)水着バージョン,490119,SSR水着ルオー,,
Joy (Event SSR),ジョイ(イベントSSR),3040588000,3,1,7,5,0,0,,false,108,1080,,1711,9011,,,,,,false,false,,,{2146},Joy (Event SSR),2025-03-11,,,ジョイ (SSR),489593,SSRジョイ,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Basara (Grand) バサラ (リミテッドver) 3040582000 3 6 10 7 1 2 false 192 1128 1740 8760 false false {3271} Basara 2025-03-17 バサラ (SSR)リミテッドバージョン 489323 SSRバサラ
3 Mahira (Summer) マキラ(水着ver) 3040584000 3 1 8 7 2 3 false 281 1217 1408 8915 false false {3073} Mahira (Summer) 2025-03-17 マキラ (SSR)水着バージョン 489328 SSR水着マキラ
4 Lu Woh (Summer) ル・オー(水着ver) 3040583000 3 4 7 6 0 2 false 236 1172 1385 7820 false false {3221} Lu Woh (Summer) 2025-03-17 ル・オー (SSR)水着バージョン 490119 SSR水着ルオー
5 Joy (Event SSR) ジョイ(イベントSSR) 3040588000 3 1 7 5 0 0 false 108 1080 1711 9011 false false {2146} Joy (Event SSR) 2025-03-11 ジョイ (SSR) 489593 SSRジョイ

View file

@ -0,0 +1,8 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
Canifortis,天干地支刀・戌之威,1040917200,3,6,10,2,true,false,150,15,38,215,259,,474,2895,3500,,false,,false,false,3040582000,,2025-03-17,2025-03-17,,Canifortis,天干地支刀・戌之威 (SSR),490121,天干地支刀・戌之威,,,false,,,,,,,,,,,,,,
Tenth Crow of the Clutch,第十酉行筒,1040517200,3,1,9,99,false,false,100,10,33,196,,,434,2606,,,false,,false,false,3040584000,,2025-03-17,,,Tenth Crow of the Clutch,第十酉行筒 (SSR),490124,第十酉行筒,,,false,,,,,,,,,,,,,,
Lu Woh Float,ル・オー・フロート,1040817100,3,4,8,99,false,false,100,10,51,303,,,344,2073,,,false,,false,false,3040583000,,2025-03-17,,,Lu Woh Float,ル・オー・フロート (SSR),490125,ル・オー・フロート,,,false,,,,,,,,,,,,,,
Exo Heliocentrum,神杖エクス・ヘリオセント,1040424300,3,6,6,39,true,false,150,15,41,260,315,,333,2029,2453,,false,,false,false,,,2025-03-22,2025-03-22,,Exo Heliocentrum,神杖エクス・ヘリオセント (SSR),490430,神杖エクス・ヘリオセント,,,false,,,,,,,,,,,,,,
Onmyoji's Reito,陰陽之霊刀,1040917300,3,0,10,19,true,true,200,20,35,,,281,430,,,3856,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Onmyoji%27s_Reito,陰陽之霊刀 (SSR),490757,陰陽之霊刀,,,false,,,,,,,,,,,,,,
Ouranosphaira Ravdos,ウラニアスフェラ・ラヴドス,1040424400,3,0,6,19,true,true,200,20,44,,,371,379,,,3390,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Ouranosphaira Ravdos,ウラニアスフェラ・ラヴドス (SSR),490756,ウラニアスフェラ・ラヴドス,,,false,,,,,,,,,,,,,,
Cometa Sica,コメーテス・シーカ,1040121300,3,0,2,19,true,true,200,20,43,,,397,384,,,3251,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Cometa Sica,コメーテス・シーカ (SSR),490753,コメーテス・シーカ,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 Canifortis 天干地支刀・戌之威 1040917200 3 6 10 2 true false 150 15 38 215 259 474 2895 3500 false false false 3040582000 2025-03-17 2025-03-17 Canifortis 天干地支刀・戌之威 (SSR) 490121 天干地支刀・戌之威 false
3 Tenth Crow of the Clutch 第十酉行筒 1040517200 3 1 9 99 false false 100 10 33 196 434 2606 false false false 3040584000 2025-03-17 Tenth Crow of the Clutch 第十酉行筒 (SSR) 490124 第十酉行筒 false
4 Lu Woh Float ル・オー・フロート 1040817100 3 4 8 99 false false 100 10 51 303 344 2073 false false false 3040583000 2025-03-17 Lu Woh Float ル・オー・フロート (SSR) 490125 ル・オー・フロート false
5 Exo Heliocentrum 神杖エクス・ヘリオセント 1040424300 3 6 6 39 true false 150 15 41 260 315 333 2029 2453 false false false 2025-03-22 2025-03-22 Exo Heliocentrum 神杖エクス・ヘリオセント (SSR) 490430 神杖エクス・ヘリオセント false
6 Onmyoji's Reito 陰陽之霊刀 1040917300 3 0 10 19 true true 200 20 35 281 430 3856 false false false 2025-03-25 2025-03-25 2025-03-25 Onmyoji%27s_Reito 陰陽之霊刀 (SSR) 490757 陰陽之霊刀 false
7 Ouranosphaira Ravdos ウラニアスフェラ・ラヴドス 1040424400 3 0 6 19 true true 200 20 44 371 379 3390 false false false 2025-03-25 2025-03-25 2025-03-25 Ouranosphaira Ravdos ウラニアスフェラ・ラヴドス (SSR) 490756 ウラニアスフェラ・ラヴドス false
8 Cometa Sica コメーテス・シーカ 1040121300 3 0 2 19 true true 200 20 43 397 384 3251 false false false 2025-03-25 2025-03-25 2025-03-25 Cometa Sica コメーテス・シーカ (SSR) 490753 コメーテス・シーカ false

View file

@ -0,0 +1,16 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
,,1040022600,,,,,true,,150,15,,,221,,,,2603,,,,,,,,,2025-03-14,,,,,,,,,,,,,,,,,,,,,,
,,1040216800,,,,,true,,150,15,,,264,,,,2380,,,,,,,,,2025-03-14,,,,,,,,,,,,,,,,,,,,,,
,,1040618900,,,,,true,,150,15,,,297,,,,2854,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040713700,,,,,true,,150,15,,,270,,,,3048,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040423400,,,,,true,,150,15,,,354,,,,2568,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040916600,,,,,true,,150,15,,,247,,,,3125,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040119100,,,,,true,,150,15,,,278,,,,2943,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040518000,,,,,true,,150,15,,,213,,,,3267,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040617200,,,,,true,,150,15,,,302,,,,2828,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040916400,,,,,true,,150,15,,,214,,,,3288,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040119600,,,,,true,,150,15,,,291,,,,2879,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040816900,,,,,true,,150,15,,,335,,,,2659,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040026000,,,,,true,,150,15,,,302,,,,2821,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040816800,,,,,true,,150,15,,,366,,,,2507,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
Decorus Sicarius,,1040121100,,,,,,,,,,,,,,,,,,,,,,,,,,Decorus Sicarius,,,,,,,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 1040022600 true 150 15 221 2603 2025-03-14
3 1040216800 true 150 15 264 2380 2025-03-14
4 1040618900 true 150 15 297 2854 2025-03-10
5 1040713700 true 150 15 270 3048 2025-03-10
6 1040423400 true 150 15 354 2568 2025-03-10
7 1040916600 true 150 15 247 3125 2025-03-10
8 1040119100 true 150 15 278 2943 2025-03-10
9 1040518000 true 150 15 213 3267 2025-03-10
10 1040617200 true 150 15 302 2828 2025-03-10
11 1040916400 true 150 15 214 3288 2025-03-10
12 1040119600 true 150 15 291 2879 2025-03-10
13 1040816900 true 150 15 335 2659 2025-03-10
14 1040026000 true 150 15 302 2821 2025-03-10
15 1040816800 true 150 15 366 2507 2025-03-10
16 Decorus Sicarius 1040121100 Decorus Sicarius

View file

@ -0,0 +1,4 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Seofon (Yukata),シエテ(浴衣ver),3040586000,3,5,1,10,1,1,,false,277,1277,,1777,9777,,,,,,false,false,,,{4007},Seofon (Yukata),2025-03-30,,,シエテ (SSR)浴衣バージョン,489325,SSR浴衣シエテ,,
Vikala (Yukata),ビカラ(浴衣ver),3040585000,3,6,3,7,2,1,,false,280,1550,,1600,8250,,,,,,false,false,,,{3150},Vikala (Yukata),2025-03-30,,,ビカラ (SSR)浴衣バージョン,491855,SSR浴衣ビカラ,,
,,3040087000,,,,,,,,true,300,1600,1900,1500,8000,9500,,,,,,,,,,,2016-06-30,2025-03-29,,ロザミア (SSR),33985,SSRロザミア,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Seofon (Yukata) シエテ(浴衣ver) 3040586000 3 5 1 10 1 1 false 277 1277 1777 9777 false false {4007} Seofon (Yukata) 2025-03-30 シエテ (SSR)浴衣バージョン 489325 SSR浴衣シエテ
3 Vikala (Yukata) ビカラ(浴衣ver) 3040585000 3 6 3 7 2 1 false 280 1550 1600 8250 false false {3150} Vikala (Yukata) 2025-03-30 ビカラ (SSR)浴衣バージョン 491855 SSR浴衣ビカラ
4 3040087000 true 300 1600 1900 1500 8000 9500 2016-06-30 2025-03-29 ロザミア (SSR) 33985 SSRロザミア

View file

@ -0,0 +1,5 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
First Fling of the Mischief,第一子行弾弓,1040714000,3,6,5,99,true,false,150,15,46,236,284,,377,2460,2981,,false,,false,false,,,2025-03-30,2025-03-30,,First Fling of the Mischief,第一子行弾弓 (SSR),,第一子行弾弓,,,false,,,,,,,,,,,,,,
Prismatic Trientalis,七彩華刀,1040917900,3,5,10,99,false,false,100,10,18,174,,,515,2737,,,false,,false,false,,,2025-03-30,,,Prismatic Trientalis,七彩華刀 (SSR),,七彩華刀,,,false,,,,,,,,,,,,,,
Scarface,スカーフェイス,1040517300,3,5,9,98,false,false,100,10,11,132,,,408,2221,,,false,,false,false,,,2025-03-29,,,Scarface,スカーフェイス (SSR),491802,スカーフェイス,,,false,,,,,,,,,,,,,,
Henchman,ヘンチマン,1030109100,2,5,2,98,false,false,75,10,8,117,,,271,1368,,,false,,false,false,,,2025-03-29,,,Henchman,ヘンチマン (SR),,ヘンチマン,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 First Fling of the Mischief 第一子行弾弓 1040714000 3 6 5 99 true false 150 15 46 236 284 377 2460 2981 false false false 2025-03-30 2025-03-30 First Fling of the Mischief 第一子行弾弓 (SSR) 第一子行弾弓 false
3 Prismatic Trientalis 七彩華刀 1040917900 3 5 10 99 false false 100 10 18 174 515 2737 false false false 2025-03-30 Prismatic Trientalis 七彩華刀 (SSR) 七彩華刀 false
4 Scarface スカーフェイス 1040517300 3 5 9 98 false false 100 10 11 132 408 2221 false false false 2025-03-29 Scarface スカーフェイス (SSR) 491802 スカーフェイス false
5 Henchman ヘンチマン 1030109100 2 5 2 98 false false 75 10 8 117 271 1368 false false false 2025-03-29 Henchman ヘンチマン (SR) ヘンチマン false

View file

@ -35,26 +35,31 @@ module Granblue
# @param verbose [Boolean] When true, enables detailed logging # @param verbose [Boolean] When true, enables detailed logging
# @param storage [Symbol] Storage mode (:local, :s3, or :both) # @param storage [Symbol] Storage mode (:local, :s3, or :both)
# @return [void] # @return [void]
def initialize(id, test_mode: false, verbose: false, storage: :both) def initialize(id, test_mode: false, verbose: false, storage: :both, logger: nil)
@id = id @id = id
@base_url = base_url @base_url = base_url
@test_mode = test_mode @test_mode = test_mode
@verbose = verbose @verbose = verbose
@storage = storage @storage = storage
@logger = logger || Logger.new($stdout) # fallback logger
@aws_service = AwsService.new @aws_service = AwsService.new
ensure_directories_exist unless @test_mode ensure_directories_exist unless @test_mode
end end
# Download images for all sizes # Download images for all sizes
# @param selected_size [String] The size to download
# @return [void] # @return [void]
def download def download(selected_size = nil)
log_info "-> #{@id}" log_info("-> #{@id}")
return if @test_mode return if @test_mode
SIZES.each_with_index do |size, index| # If a specific size is provided, use only that; otherwise, use all available sizes.
sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size) path = download_path(size)
url = build_url(size) url = build_url(size)
process_download(url, size, path, last: index == SIZES.size - 1) process_download(url, size, path, last: index == sizes.size - 1)
end end
end end
@ -128,10 +133,10 @@ module Granblue
download.rewind download.rewind
# Upload to S3 if it doesn't exist # Upload to S3 if it doesn't exist
unless @aws_service.file_exists?(s3_key) return if @aws_service.file_exists?(s3_key)
@aws_service.upload_stream(download, s3_key) @aws_service.upload_stream(download, s3_key)
end end
end
# Check if file should be downloaded based on storage mode # Check if file should be downloaded based on storage mode
# @param local_path [String] Local file path # @param local_path [String] Local file path
@ -182,7 +187,7 @@ module Granblue
# Log informational message if verbose # Log informational message if verbose
# @param message [String] Message # @param message [String] Message
def log_info(message) def log_info(message)
puts message if @verbose @logger.info(message) if @verbose
end end
# Download elemental variant image # Download elemental variant image
@ -197,13 +202,11 @@ module Granblue
filepath = "#{path}/#{filename}" filepath = "#{path}/#{filename}"
URI.open(url) do |file| URI.open(url) do |file|
content = file.read content = file.read
if content raise "Failed to read content from #{url}" unless content
File.open(filepath, 'wb') do |output| File.open(filepath, 'wb') do |output|
output.write(content) output.write(content)
end end
else
raise "Failed to read content from #{url}"
end
end end
log_info "-> #{size}:\t#{url}..." log_info "-> #{size}:\t#{url}..."
rescue OpenURI::HTTPError rescue OpenURI::HTTPError

View file

@ -15,24 +15,27 @@ module Granblue
# Downloads images for all variants of a character based on their uncap status. # Downloads images for all variants of a character based on their uncap status.
# Overrides {BaseDownloader#download} to handle character-specific variants. # Overrides {BaseDownloader#download} to handle character-specific variants.
# #
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Skips download if character is not found in database # @note Skips download if character is not found in database
# @note Downloads FLB/ULB variants only if character has those uncaps # @note Downloads FLB/ULB variants only if character has those uncaps
# @see #download_variants # @see #download_variants
def download def download(selected_size = nil)
character = Character.find_by(granblue_id: @id) character = Character.find_by(granblue_id: @id)
return unless character return unless character
download_variants(character) download_variants(character, selected_size)
end end
private private
# Downloads all variants of a character's images # Downloads all variants of a character's images
#
# @param character [Character] Character model instance to download images for # @param character [Character] Character model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Only downloads variants that should exist based on character uncap status # @note Only downloads variants that should exist based on character uncap status
def download_variants(character) def download_variants(character, selected_size = nil)
# All characters have 01 and 02 variants # All characters have 01 and 02 variants
variants = %W[#{@id}_01 #{@id}_02] variants = %W[#{@id}_01 #{@id}_02]
@ -45,18 +48,22 @@ module Granblue
log_info "Downloading character variants: #{variants.join(', ')}" if @verbose log_info "Downloading character variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id| variants.each do |variant_id|
download_variant(variant_id) download_variant(variant_id, selected_size)
end end
end end
# Downloads a specific variant's images in all sizes # Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Character variant ID (e.g., "3040001000_01") # @param variant_id [String] Character variant ID (e.g., "3040001000_01")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
def download_variant(variant_id) def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose log_info "-> #{variant_id}" if @verbose
return if @test_mode return if @test_mode
SIZES.each_with_index do |size, index| sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size) path = download_path(size)
url = build_variant_url(variant_id, size) url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1) process_download(url, size, path, last: index == SIZES.size - 1)
@ -64,13 +71,19 @@ module Granblue
end end
# Builds URL for a specific variant and size # Builds URL for a specific variant and size
#
# @param variant_id [String] Character variant ID # @param variant_id [String] Character variant ID
# @param size [String] Image size variant ("main", "grid", or "square") # @param size [String] Image size variant ("main", "grid", "square", or "detail")
# @return [String] Complete URL for downloading the image # @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size) def build_variant_url(variant_id, size)
directory = directory_for_size(size) directory = directory_for_size(size)
if size == 'detail'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg" "#{@base_url}/#{directory}/#{variant_id}.jpg"
end end
end
# Gets object type for file paths and storage keys # Gets object type for file paths and storage keys
# @return [String] Returns "character" # @return [String] Returns "character"
@ -85,6 +98,7 @@ module Granblue
end end
# Gets directory name for a size variant # Gets directory name for a size variant
#
# @param size [String] Image size variant # @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure # @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "f", "grid" -> "m", "square" -> "s" # @note Maps "main" -> "f", "grid" -> "m", "square" -> "s"
@ -93,6 +107,7 @@ module Granblue
when 'main' then 'f' when 'main' then 'f'
when 'grid' then 'm' when 'grid' then 'm'
when 'square' then 's' when 'square' then 's'
when 'detail' then 'detail'
end end
end end
end end

View file

@ -15,25 +15,28 @@ module Granblue
# Downloads images for all variants of a summon based on their uncap status. # Downloads images for all variants of a summon based on their uncap status.
# Overrides {BaseDownloader#download} to handle summon-specific variants. # Overrides {BaseDownloader#download} to handle summon-specific variants.
# #
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Skips download if summon is not found in database # @note Skips download if summon is not found in database
# @note Downloads ULB and transcendence variants only if summon has those uncaps # @note Downloads ULB and transcendence variants only if summon has those uncaps
# @see #download_variants # @see #download_variants
def download def download(selected_size = nil)
summon = Summon.find_by(granblue_id: @id) summon = Summon.find_by(granblue_id: @id)
return unless summon return unless summon
download_variants(summon) download_variants(summon, selected_size)
end end
private private
# Downloads all variants of a summon's images # Downloads all variants of a summon's images
#
# @param summon [Summon] Summon model instance to download images for # @param summon [Summon] Summon model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Only downloads variants that should exist based on summon uncap status # @note Only downloads variants that should exist based on summon uncap status
# @note Handles special transcendence art variants for 6★ summons # @note Handles special transcendence art variants for 6★ summons
def download_variants(summon) def download_variants(summon, selected_size = nil)
# All summons have base variant # All summons have base variant
variants = [@id] variants = [@id]
@ -41,26 +44,28 @@ module Granblue
variants << "#{@id}_02" if summon.ulb variants << "#{@id}_02" if summon.ulb
# Add Transcendence variants if available # Add Transcendence variants if available
if summon.transcendence variants.push("#{@id}_03", "#{@id}_04") if summon.transcendence
variants.push("#{@id}_03", "#{@id}_04")
end
log_info "Downloading summon variants: #{variants.join(', ')}" if @verbose log_info "Downloading summon variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id| variants.each do |variant_id|
download_variant(variant_id) download_variant(variant_id, selected_size)
end end
end end
# Downloads a specific variant's images in all sizes # Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Summon variant ID (e.g., "2040001000_02") # @param variant_id [String] Summon variant ID (e.g., "2040001000_02")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Downloads all size variants (main/grid/square) for the given variant # @note Downloads all size variants (main/grid/square) for the given variant
def download_variant(variant_id) def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose log_info "-> #{variant_id}" if @verbose
return if @test_mode return if @test_mode
SIZES.each_with_index do |size, index| sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size) path = download_path(size)
url = build_variant_url(variant_id, size) url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1) process_download(url, size, path, last: index == SIZES.size - 1)
@ -68,13 +73,18 @@ module Granblue
end end
# Builds URL for a specific variant and size # Builds URL for a specific variant and size
#
# @param variant_id [String] Summon variant ID # @param variant_id [String] Summon variant ID
# @param size [String] Image size variant ("main", "grid", or "square") # @param size [String] Image size variant ("main", "grid", "square", or "detail")
# @return [String] Complete URL for downloading the image # @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size) def build_variant_url(variant_id, size)
directory = directory_for_size(size) directory = directory_for_size(size)
if size == 'detail'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg" "#{@base_url}/#{directory}/#{variant_id}.jpg"
end end
end
# Gets object type for file paths and storage keys # Gets object type for file paths and storage keys
# @return [String] Returns "summon" # @return [String] Returns "summon"
@ -89,14 +99,16 @@ module Granblue
end end
# Gets directory name for a size variant # Gets directory name for a size variant
#
# @param size [String] Image size variant # @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure # @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "party_main", "grid" -> "party_sub", "square" -> "s" # @note Maps "main" -> "party_main", "grid" -> "party_sub", "square" -> "s"
def directory_for_size(size) def directory_for_size(size)
case size.to_s case size.to_s
when 'main' then 'party_main' when 'main' then 'ls'
when 'grid' then 'party_sub' when 'grid' then 'm'
when 'square' then 's' when 'square' then 's'
when 'detail' then 'detail'
end end
end end
end end

View file

@ -16,49 +16,54 @@ module Granblue
# Downloads images for all variants of a weapon based on their uncap status. # Downloads images for all variants of a weapon based on their uncap status.
# Overrides {BaseDownloader#download} to handle weapon-specific variants. # Overrides {BaseDownloader#download} to handle weapon-specific variants.
# #
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Skips download if weapon is not found in database # @note Skips download if weapon is not found in database
# @note Downloads transcendence variants only if weapon has those uncaps # @note Downloads transcendence variants only if weapon has those uncaps
# @see #download_variants # @see #download_variants
def download def download(selected_size = nil)
weapon = Weapon.find_by(granblue_id: @id) weapon = Weapon.find_by(granblue_id: @id)
return unless weapon return unless weapon
download_variants(weapon) download_variants(weapon, selected_size)
end end
private private
# Downloads all variants of a weapon's images # Downloads all variants of a weapon's images
#
# @param weapon [Weapon] Weapon model instance to download images for # @param weapon [Weapon] Weapon model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Only downloads variants that should exist based on weapon uncap status # @note Only downloads variants that should exist based on weapon uncap status
# @note Handles special transcendence art variants for transcendable weapons # @note Handles special transcendence art variants for transcendable weapons
def download_variants(weapon) def download_variants(weapon, selected_size = nil)
# All weapons have base variant # All weapons have base variant
variants = [@id] variants = [@id]
# Add transcendence variants if available # Add transcendence variants if available
if weapon.transcendence variants.push("#{@id}_02", "#{@id}_03") if weapon.transcendence
variants.push("#{@id}_02", "#{@id}_03")
end
log_info "Downloading weapon variants: #{variants.join(', ')}" if @verbose log_info "Downloading weapon variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id| variants.each do |variant_id|
download_variant(variant_id) download_variant(variant_id, selected_size)
end end
end end
# Downloads a specific variant's images in all sizes # Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Weapon variant ID (e.g., "1040001000_02") # @param variant_id [String] Weapon variant ID (e.g., "1040001000_02")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void] # @return [void]
# @note Downloads all size variants (main/grid/square) for the given variant # @note Downloads all size variants (main/grid/square) for the given variant
def download_variant(variant_id) def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose log_info "-> #{variant_id}" if @verbose
return if @test_mode return if @test_mode
SIZES.each_with_index do |size, index| sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size) path = download_path(size)
url = build_variant_url(variant_id, size) url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1) process_download(url, size, path, last: index == SIZES.size - 1)
@ -66,13 +71,18 @@ module Granblue
end end
# Builds URL for a specific variant and size # Builds URL for a specific variant and size
#
# @param variant_id [String] Weapon variant ID # @param variant_id [String] Weapon variant ID
# @param size [String] Image size variant ("main", "grid", or "square") # @param size [String] Image size variant ("main", "grid", "square", or "raw")
# @return [String] Complete URL for downloading the image # @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size) def build_variant_url(variant_id, size)
directory = directory_for_size(size) directory = directory_for_size(size)
if size == 'raw'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg" "#{@base_url}/#{directory}/#{variant_id}.jpg"
end end
end
# Gets object type for file paths and storage keys # Gets object type for file paths and storage keys
# @return [String] Returns "weapon" # @return [String] Returns "weapon"
@ -87,6 +97,7 @@ module Granblue
end end
# Gets directory name for a size variant # Gets directory name for a size variant
#
# @param size [String] Image size variant # @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure # @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "ls", "grid" -> "m", "square" -> "s" # @note Maps "main" -> "ls", "grid" -> "m", "square" -> "s"
@ -95,6 +106,7 @@ module Granblue
when 'main' then 'ls' when 'main' then 'ls'
when 'grid' then 'm' when 'grid' then 'm'
when 'square' then 's' when 'square' then 's'
when 'raw' then 'b'
end end
end end
end end

View file

@ -4,21 +4,26 @@ require 'pry'
module Granblue module Granblue
module Parsers module Parsers
# CharacterParser parses character data from gbf.wiki # CharacterParser parses character data from gbf.wiki
class CharacterParser class CharacterParser
attr_reader :granblue_id attr_reader :granblue_id
def initialize(granblue_id: String, debug: false) def initialize(granblue_id: String, debug: false, use_local: false)
@character = Character.find_by(granblue_id: granblue_id) @character = Character.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new @wiki = Granblue::Parsers::Wiki.new
@debug = debug || false @debug = debug || false
@use_local = use_local
end end
# Fetches using @wiki and then processes the response # Fetches using @wiki and then processes the response
# Returns true if successful, false if not # Returns true if successful, false if not
# Raises an exception if something went wrong # Raises an exception if something went wrong
def fetch(save: false) def fetch(save: false)
if @use_local && @character.wiki_raw.present?
wikitext = @character.wiki_raw
return handle_fetch_success(wikitext, save)
end
response = fetch_wiki_info response = fetch_wiki_info
return false if response.nil? return false if response.nil?
@ -49,6 +54,9 @@ module Granblue
# Handle the response from the wiki if the response is successful # Handle the response from the wiki if the response is successful
# If the save flag is set, it will persist the data to the database # If the save flag is set, it will persist the data to the database
def handle_fetch_success(response, save) def handle_fetch_success(response, save)
@character.wiki_raw = response
@character.save!
ap "#{@character.granblue_id}: Successfully fetched info for #{@character.wiki_en}" if @debug ap "#{@character.granblue_id}: Successfully fetched info for #{@character.wiki_en}" if @debug
extracted = parse_string(response) extracted = parse_string(response)
info = parse(extracted) info = parse(extracted)
@ -152,12 +160,12 @@ module Granblue
info[:id] = hash['id'] info[:id] = hash['id']
info[:charid] = hash['charid'].scan(/\b\d{4}\b/) info[:charid] = hash['charid'].scan(/\b\d{4}\b/)
info[:flb] = GranblueWiki.boolean.fetch(hash['5star'], false) info[:flb] = Granblue::Parsers::Wiki.boolean.fetch(hash['5star'], false)
info[:ulb] = hash['max_evo'].to_i == 6 info[:ulb] = hash['max_evo'].to_i == 6
info[:rarity] = GranblueWiki.rarities.fetch(hash['rarity'], 0) info[:rarity] = Granblue::Parsers::Wiki.rarities.fetch(hash['rarity'], 0)
info[:element] = GranblueWiki.elements.fetch(hash['element'], 0) info[:element] = Granblue::Parsers::Wiki.elements.fetch(hash['element'], 0)
info[:gender] = GranblueWiki.genders.fetch(hash['gender'], 0) info[:gender] = Granblue::Parsers::Wiki.genders.fetch(hash['gender'], 0)
info[:proficiencies] = proficiencies_from_hash(hash['weapon']) info[:proficiencies] = proficiencies_from_hash(hash['weapon'])
info[:races] = races_from_hash(hash['race']) info[:races] = races_from_hash(hash['race'])
@ -211,14 +219,14 @@ module Granblue
# Converts proficiencies from a string to a hash # Converts proficiencies from a string to a hash
def proficiencies_from_hash(character) def proficiencies_from_hash(character)
character.to_s.split(',').map.with_index do |prof, i| character.to_s.split(',').map.with_index do |prof, i|
{ "proficiency#{i + 1}" => GranblueWiki.proficiencies[prof] } { "proficiency#{i + 1}" => Granblue::Parsers::Wiki.proficiencies[prof] }
end.reduce({}, :merge) end.reduce({}, :merge)
end end
# Converts races from a string to a hash # Converts races from a string to a hash
def races_from_hash(race) def races_from_hash(race)
race.to_s.split(',').map.with_index do |r, i| race.to_s.split(',').map.with_index do |r, i|
{ "race#{i + 1}" => GranblueWiki.races[r] } { "race#{i + 1}" => Granblue::Parsers::Wiki.races[r] }
end.reduce({}, :merge) end.reduce({}, :merge)
end end

View file

@ -10,7 +10,7 @@ module Granblue
def initialize(granblue_id: String, debug: false) def initialize(granblue_id: String, debug: false)
@summon = Summon.find_by(granblue_id: granblue_id) @summon = Summon.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new(debug: debug) @wiki = Granblue::Parsers::Wiki.new
@debug = debug || false @debug = debug || false
end end

View file

@ -10,7 +10,7 @@ module Granblue
def initialize(granblue_id: String, debug: false) def initialize(granblue_id: String, debug: false)
@weapon = Weapon.find_by(granblue_id: granblue_id) @weapon = Weapon.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new(debug: debug) @wiki = Granblue::Parsers::Wiki.new
@debug = debug || false @debug = debug || false
end end
@ -278,17 +278,17 @@ module Granblue
# Converts rarities from a string to a hash # Converts rarities from a string to a hash
def rarity_from_hash(string) def rarity_from_hash(string)
string ? GranblueWiki.rarities[string.upcase] : nil string ? Granblue::Parsers::Wiki.rarities[string.upcase] : nil
end end
# Converts proficiencies from a string to a hash # Converts proficiencies from a string to a hash
def proficiency_from_hash(string) def proficiency_from_hash(string)
GranblueWiki.proficiencies[string] Granblue::Parsers::Wiki.proficiencies[string]
end end
# Converts a bullet type from a string to a hash # Converts a bullet type from a string to a hash
def bullet_from_hash(string) def bullet_from_hash(string)
string ? GranblueWiki.bullets[string] : nil string ? Granblue::Parsers::Wiki.bullets[string] : nil
end end
# Parses a date string into a Date object # Parses a date string into a Date object

42
lib/tasks/database.rake Normal file
View file

@ -0,0 +1,42 @@
namespace :db do
desc 'Backup remote PostgreSQL database'
task :backup do
remote_host = ENV.fetch('REMOTE_DB_HOST', 'roundhouse.proxy.rlwy.net')
remote_port = ENV.fetch('REMOTE_DB_PORT', '54629')
remote_user = ENV.fetch('REMOTE_DB_USER', 'postgres')
remote_db = ENV.fetch('REMOTE_DB_NAME', 'railway')
password = ENV.fetch('REMOTE_DB_PASSWORD') { raise 'Please set REMOTE_DB_PASSWORD' }
backup_dir = File.expand_path('backups')
FileUtils.mkdir_p(backup_dir)
backup_file = File.join(backup_dir, "#{Time.now.strftime('%Y%m%d_%H%M%S')}-prod-backup.tar")
cmd = %W[
pg_dump -h #{remote_host} -p #{remote_port} -U #{remote_user} -d #{remote_db} -F t
--no-owner --exclude-extension=timescaledb --exclude-extension=timescaledb_toolkit
].join(' ')
puts "Backing up remote database to #{backup_file}..."
system({ 'PGPASSWORD' => password }, "#{cmd} > #{backup_file}")
puts 'Backup completed!'
end
desc 'Restore PostgreSQL database from backup'
task :restore, [:backup_file] => [:environment] do |_, args|
local_user = ENV.fetch('LOCAL_DB_USER', 'justin')
local_db = ENV.fetch('LOCAL_DB_NAME', 'hensei_dev')
# Use the specified backup file or find the most recent one
backup_dir = File.expand_path('backups')
backup_file = args[:backup_file] || Dir.glob("#{backup_dir}/*-prod-backup.tar").max
raise 'Backup file not found. Please specify a valid backup file.' unless backup_file && File.exist?(backup_file)
puts "Restoring database from #{backup_file}..."
system("pg_restore --no-owner --role=#{local_user} --disable-triggers -U #{local_user} -d #{local_db} #{backup_file}")
puts 'Restore completed!'
end
desc 'Backup remote database and restore locally'
task backup_and_restore: %i[backup restore]
end

View file

@ -1,40 +1,50 @@
namespace :granblue do namespace :granblue do
def _progress_reporter(count:, total:, result:, bar_len: 40, multi: true) desc 'Downloads all images for the given object type'
filled_len = (bar_len * count / total).round # Downloads all images for a specific type of game object (e.g. summons, weapons)
status = File.basename(result) # Uses the appropriate downloader class based on the object type
percents = (100.0 * count / total).round(1) #
bar = '=' * filled_len + '-' * (bar_len - filled_len) # @param object [String] Type of object to download images for (e.g. 'summon', 'weapon')
# @example Download all summon images
# rake granblue:download_all_images\[summon\]
# @example Download all weapon images
# rake granblue:download_all_images\[weapon\]
# @example Download all character images
# rake granblue:download_all_images\[character\]
task :download_all_images, %i[object threads size] => :environment do |_t, args|
require 'parallel'
require 'logger'
if !multi # Use a thread-safe logger (or Rails.logger if preferred)
print("[#{bar}] #{percents}% ...#{' ' * 14}#{status}\n") logger = Logger.new($stdout)
logger.level = Logger::INFO # set to WARN or INFO to reduce debug noise
# Load downloader classes
require_relative '../granblue/downloaders/base_downloader'
Dir[Rails.root.join('lib', 'granblue', 'downloaders', '*.rb')].each { |file| require file }
object = args[:object]
specified_size = args[:size]
klass = object.classify.constantize
ids = klass.pluck(:granblue_id)
puts "Downloading images for #{ids.count} #{object.pluralize}..."
logger.info "Downloading images for #{ids.count} #{object.pluralize}..."
thread_count = (args[:threads] || 4).to_i
logger.info "Using #{thread_count} threads for parallel downloads..."
logger.info "Downloading only size: #{specified_size}" if specified_size
Parallel.each(ids, in_threads: thread_count) do |id|
ActiveRecord::Base.connection_pool.with_connection do
downloader_class = "Granblue::Downloaders::#{object.classify}Downloader".constantize
downloader = downloader_class.new(id, verbose: true, logger: logger)
if specified_size
downloader.download(specified_size)
else else
print "\n" downloader.download
end
end
desc 'Downloads images for the given object type at the given size'
task :download_all_images, %i[object size] => :environment do |_t, args|
require 'open-uri'
filename = "export/#{args[:object]}-#{args[:size]}.txt"
count = `wc -l #{filename}`.split.first.to_i
path = "#{Rails.root}/download/#{args[:object]}-#{args[:size]}"
FileUtils.mkdir_p(path) unless Dir.exist?(path)
puts "Downloading #{count} images from #{args[:object]}-#{args[:size]}.txt..."
if File.exist?(filename)
File.readlines(filename).each_with_index do |line, i|
download = URI.parse(line.strip).open
download_URI = "#{path}/#{download.base_uri.to_s.split('/')[-1]}"
if File.exist?(download_URI)
puts "Skipping #{line}"
else
IO.copy_stream(download, "#{path}/#{download.base_uri.to_s.split('/')[-1]}")
_progress_reporter(count: i, total: count, result: download_URI, bar_len: 40, multi: false)
end end
rescue StandardError => e rescue StandardError => e
puts "#{e}: #{line}" logger.error "Error downloading #{object} #{id}: #{e.message}"
end end
end end
end end

79
lib/tasks/fetch_wiki.rake Normal file
View file

@ -0,0 +1,79 @@
namespace :granblue do
desc <<~DESC
Fetch and store raw wiki data for objects (Character, Weapon, Summon).
Usage:
rake granblue:fetch_wiki_data # Fetch all Characters (default)
rake granblue:fetch_wiki_data type=Weapon # Fetch all Weapons
rake granblue:fetch_wiki_data type=Summon # Fetch all Summons
rake granblue:fetch_wiki_data type=Character id=5 # Fetch specific Character by ID
rake granblue:fetch_wiki_data force=true # Force re-download even if data exists
DESC
task fetch_wiki_data: :environment do
# Get parameters from environment
type = (ENV['type'] || 'Character').classify
id = ENV['id']
force = ENV['force'] == 'true'
# Validate object type
valid_types = %w[Character Weapon Summon]
unless valid_types.include?(type)
puts "Error: Invalid type '#{type}'. Must be one of: #{valid_types.join(', ')}"
exit 1
end
# Get the class from the type string
klass = type.constantize
# Setup query - either all objects or specific one
query = id.present? ? klass.where(granblue_id: id) : klass.all
errors = []
count = 0
query.find_each do |object|
# Skip objects that already have wiki_raw if force is not set
if object.wiki_raw.present? && !force
puts "Skipping #{object.name_en} (already has wiki_raw)."
next
end
# If the object doesn't have a wiki page specified, skip
if object.wiki_en.blank?
puts "Skipping #{object.name_en} (no wiki_en set)."
next
end
begin
# 1) Fetch raw wikitext from the wiki
wiki_text = Granblue::Parsers::Wiki.new.fetch(object.wiki_en)
# 2) Check if the page is a redirect
redirect_match = wiki_text.match(/#REDIRECT \[\[(.*?)\]\]/)
if redirect_match
redirect_target = redirect_match[1]
# Update object to new wiki_en so we don't keep fetching the old page
object.update!(wiki_en: redirect_target)
# Fetch again with the new page name
wiki_text = Granblue::Parsers::Wiki.new.fetch(redirect_target)
end
puts wiki_text
# 3) Save raw wiki text in the object record
object.update!(wiki_raw: wiki_text)
puts "Saved wiki data for #{object.name_en} (#{object.id})"
count += 1
rescue StandardError => e
errors << { object_id: object.id, type: type, error: e.message }
puts "Error fetching data for #{object.name_en}: #{e.message}"
end
end
if errors.any?
puts "#{errors.size} #{type.pluralize} had errors:"
errors.each { |err| puts " - #{err[:type]} ##{err[:object_id]} => #{err[:error]}" }
else
puts "Wiki data fetch complete for #{count} #{type.pluralize} with no errors!"
end
end
end

View file

@ -0,0 +1,17 @@
[program:hensei-api]
command=/opt/homebrew/bin/mise run s
process_name=%(program_name)s
numprocs=1
directory=/Users/justin/Developer/Granblue/hensei-api
environment=HOME="/Users/justin",MISE_CONFIG_ROOT="/Users/justin/Developer/Granblue/hensei-api",RAILS_ENV="development"
autostart=true
autorestart=unexpected
stopsignal=TERM
user=justin
stdout_logfile=/Users/justin/Developer/Granblue/hensei-api/logs/hensei-api.stdout.log
stdout_logfile_maxbytes=500KB
stdout_logfile_backups=10
stderr_logfile=/Users/justin/Developer/Granblue/hensei-api/logs/hensei-api.stderr.log
stderr_logfile_maxbytes=500KB
stderr_logfile_backups=10
serverurl=AUTO