Compare commits

..

29 commits

Author SHA1 Message Date
b03d5e6618
Fix error in weapons (#199) 2025-03-30 21:08:04 -07:00
309a499446
Merge pull request #198 from jedmund/jedmund/202503-legfest
Adds items from the March 2025 Legend Festival/Event
2025-03-30 21:05:39 -07:00
832bf86d47 New items from March Legfest and Scenario Event 2025-03-31 00:02:25 -04:00
65a58d8b4c
Merge pull request #197 from jedmund/jedmund/20250327-new-items
Add new items from March 2025
2025-03-26 22:44:36 -07:00
92de40bbbf Add columns for Classic II and Collab Gacha 2025-03-27 01:37:53 -04:00
aaa046c01f Add new items 2025-03-27 01:37:42 -04:00
2f04a7d3a7
Add a data miner for downloading data from GBF (#196)
* First pass at dataminer service

* Got output printing from dataminer

* Fetches summons, characters and weapons

* Can loop over objects

* Finish dataminer

Adds logger and continuing downloads
2025-03-06 19:12:44 -08:00
7880ac76cc
Add data ingestion endpoints (#195)
* Update schema.rb

* Add endpoints for importing game data

This lets privileged users import canonical data for characters, weapons and summons directly from the game
2025-03-02 17:48:35 -08:00
28a6b1894e
Delete db/migrate/20250301143956_add_wiki_raw_to_characters.rb (#194) 2025-03-02 16:29:05 -08:00
3746ee9af6
Merge pull request #193 from jedmund/jedmund/wiki-fields
Implement columns for raw wiki/game data
2025-03-02 16:20:55 -08:00
cec6132823 Update wiki namespace in parsers 2025-03-02 16:19:13 -08:00
311c218863 Adds a task for fetching data from wiki
Can be used to fetch one object or multiple
2025-03-02 16:17:32 -08:00
c060a4525b Update migration
Now this adds raw data columns to weapons, summons, and characters
2025-03-02 16:08:35 -08:00
2b8dfe9e20 Merge branch 'main' into jedmund/wiki-fields 2025-03-01 07:32:17 -08:00
b1800f411f
Cleans up data on some Characters (#192)
This should update the `wiki_en` field for Kaguya, Fenie, and Uriel. It also adds `character_id` for 6 characters that were missing a value.
2025-03-01 07:31:49 -08:00
2de10d03f3 Fix path for Wiki object 2025-03-01 06:34:29 -08:00
a6ede6ecf7
Merge pull request #191 from jedmund/jedmund/parallelize-downloaders
Parallelize downloaders and update sizes
2025-03-01 05:46:08 -08:00
e75578bea3 Refactor download_all_images task
This refactor focuses on implementing parallelization. This allows us to pass in a number of threads and download concurrently. This makes downloading lots of images a lot faster.
2025-03-01 05:43:57 -08:00
ffbc8d0c1e Add support for passing in a Logger 2025-03-01 05:43:05 -08:00
0d997d6ad5 Add new image sizes
* Weapons can now download the “raw”image size, which is the weapon art without a background
* Characters now download the “detail” image size, which is a horizontal crop of the character’s art
* Summons now download the “detail” image size, which is a horizontal crop of the summon’s art
* Summons also download “ls” and “m” instead of “party_main” and “party_sub”, as they match the aspect ratio of weapon sizes better, which should make our lives a lot easier.
2025-03-01 05:42:40 -08:00
5955ef2650 Add support for single sizes
You can now pass one of the sizes in to only download that size for the object.
2025-03-01 05:40:34 -08:00
ae62d594a8 Add parallel gem 2025-03-01 05:29:21 -08:00
82b3d0ed88
Merge pull request #190 from jedmund/jedmund/add-recruits
Adds recruits value to new weapons
2025-03-01 05:25:15 -08:00
e1d983a6d4 Adds recruits IDs to new weapons 2025-02-28 18:24:15 -08:00
4d3c1a800b
Update config files (#189)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Update .gitignore

Hide backups and logs directories, since we’ll be storing these in the project folder. Also hide mise’s .local directory.

* Change NewRelic log directory

Moved from log/ to logs/

* Add rake task for backing up/restoring prod db

* Rubocop fixes

* Fix error where :preview_state didn’t have an attribute

* Add supervisord ini

This uses my local paths, so we should try to abstract that away later.

* Ignore mise.toml
2025-02-27 23:13:57 -08:00
b2d2952b35
Fix limit column in weapon migration (#188)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Add items from late 2025-02

* Yuel (Grand)
* Tsukuyomi
* Sennen Goji
* Nightgaze Gale
* Bane of Avidya
* Klesha-Cleaning Dharmachakra

* Add default value for limit column
2025-02-27 20:41:51 -08:00
6bcbc97566
Jedmund/202502 update (#187)
* Update weapon series migration

This update fixes MigrateWeaponSeries from 20250218 such that it can be run on an empty database without throwing errors.

* Add items from late 2025-02

* Yuel (Grand)
* Tsukuyomi
* Sennen Goji
* Nightgaze Gale
* Bane of Avidya
* Klesha-Cleaning Dharmachakra
2025-02-27 20:27:43 -08:00
9827658771
Render the correct view when updating jobs (#186)
We were referencing an old view in `PartiesController#update_job`. This has been corrected.
2025-02-25 09:11:13 -05:00
505176ae5f
Update api URL (#185)
This change updates the production API URL to https://api.granblue.team/v1 instead of https://api.granblue.team/api/v1.
2025-02-25 08:23:47 -05:00
36 changed files with 824 additions and 150 deletions

6
.gitignore vendored
View file

@ -37,9 +37,12 @@
# Ignore master key for decrypting credentials and more.
/config/master.key
# Ignore exported and downloaded files
# Ignore specific directories
/.local
/export
/download
/backups
/logs
.DS_Store
@ -55,3 +58,4 @@ config/application.yml
# Ignore AI Codebase-generated files
codebase.md
mise.toml

View file

@ -76,10 +76,13 @@ gem 'strscan'
# New Relic Ruby Agent
gem 'newrelic_rpm'
# Parallel processing made simple and fast
gem 'parallel'
# The Sentry SDK for Rails
gem "stackprof"
gem "sentry-ruby"
gem "sentry-rails"
gem 'sentry-rails'
gem 'sentry-ruby'
gem 'stackprof'
group :doc do
gem 'apipie-rails'

View file

@ -482,6 +482,7 @@ DEPENDENCIES
mini_magick
newrelic_rpm
oj
parallel
pg
pg_query
pg_search

View file

@ -131,6 +131,7 @@ module Api
fields :edit_key
end
# Remixed view
view :remixed do
include_view :created
include_view :source_party

View file

@ -27,6 +27,8 @@ module Api
6 => 5
}.freeze
before_action :ensure_admin_role, only: %i[weapons summons characters]
##
# Processes an import request.
#
@ -49,9 +51,9 @@ module Api
end
unless raw_params['deck'].is_a?(Hash) &&
raw_params['deck'].key?('pc') &&
raw_params['deck'].key?('npc')
Rails.logger.error "[IMPORT] Deck data incomplete or missing."
raw_params['deck'].key?('pc') &&
raw_params['deck'].key?('npc')
Rails.logger.error '[IMPORT] Deck data incomplete or missing.'
return render json: { error: 'Invalid deck data' }, status: :unprocessable_content
end
@ -68,8 +70,111 @@ module Api
render json: { error: e.message }, status: :unprocessable_content
end
def weapons
Rails.logger.info '[IMPORT] Checking weapon gamedata input...'
body = parse_request_body
return unless body
weapon = Weapon.find_by(granblue_id: body['id'])
unless weapon
Rails.logger.error "[IMPORT] Weapon not found with ID: #{body['id']}"
return render json: { error: 'Weapon not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
weapon.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Weapon gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update weapon gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
def summons
Rails.logger.info '[IMPORT] Checking summon gamedata input...'
body = parse_request_body
return unless body
summon = Summon.find_by(granblue_id: body['id'])
unless summon
Rails.logger.error "[IMPORT] Summon not found with ID: #{body['id']}"
return render json: { error: 'Summon not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
summon.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Summon gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update summon gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
##
# Updates character gamedata from JSON blob.
#
# @return [void] Renders JSON response with success or error message.
def characters
Rails.logger.info '[IMPORT] Checking character gamedata input...'
body = parse_request_body
return unless body
character = Character.find_by(granblue_id: body['id'])
unless character
Rails.logger.error "[IMPORT] Character not found with ID: #{body['id']}"
return render json: { error: 'Character not found' }, status: :not_found
end
lang = params[:lang]
unless %w[en jp].include?(lang)
Rails.logger.error "[IMPORT] Invalid language: #{lang}"
return render json: { error: 'Invalid language' }, status: :unprocessable_content
end
begin
character.update!(
"game_raw_#{lang}" => body.to_json
)
render json: { message: 'Character gamedata updated successfully' }, status: :ok
rescue StandardError => e
Rails.logger.error "[IMPORT] Failed to update character gamedata: #{e.message}"
render json: { error: e.message }, status: :unprocessable_content
end
end
private
##
# Ensures the current user has admin role (role 9).
# Renders an error if the user is not an admin.
#
# @return [void]
def ensure_admin_role
return if current_user&.role == 9
Rails.logger.error "[IMPORT] Unauthorized access attempt by user #{current_user&.id}"
render json: { error: 'Unauthorized' }, status: :unauthorized
end
##
# Reads and parses the raw JSON request body.
#

View file

@ -34,10 +34,10 @@ module Api
# Remove extra subskills if necessary
if old_job &&
%w[1 2 3].include?(old_job.row) &&
%w[4 5 ex2].include?(job.row) &&
@party.skill1 && @party.skill2 && @party.skill3 &&
@party.skill1.sub && @party.skill2.sub && @party.skill3.sub
%w[1 2 3].include?(old_job.row) &&
%w[4 5 ex2].include?(job.row) &&
@party.skill1 && @party.skill2 && @party.skill3 &&
@party.skill1.sub && @party.skill2.sub && @party.skill3.sub
@party['skill3_id'] = nil
end
else
@ -47,7 +47,7 @@ module Api
end
end
render json: PartyBlueprint.render(@party, view: :jobs) if @party.save!
render json: PartyBlueprint.render(@party, view: :job_metadata) if @party.save!
end
def update_job_skills

View file

@ -78,6 +78,7 @@ class Party < ApplicationRecord
include GranblueEnums
# Define preview_state as an enum.
attribute :preview_state, :integer
enum :preview_state, { pending: 0, queued: 1, in_progress: 2, generated: 3, failed: 4 }
# ActiveRecord Associations
@ -400,7 +401,7 @@ class Party < ApplicationRecord
# @return [void]
def skills_are_unique
validate_uniqueness_of_associations([skill0, skill1, skill2, skill3],
[:skill0, :skill1, :skill2, :skill3],
%i[skill0 skill1 skill2 skill3],
:job_skills)
end
@ -410,7 +411,7 @@ class Party < ApplicationRecord
# @return [void]
def guidebooks_are_unique
validate_uniqueness_of_associations([guidebook1, guidebook2, guidebook3],
[:guidebook1, :guidebook2, :guidebook3],
%i[guidebook1 guidebook2 guidebook3],
:guidebooks)
end
@ -438,7 +439,7 @@ class Party < ApplicationRecord
def update_element!
main_weapon = weapons.detect { |gw| gw.position.to_i == -1 }
new_element = main_weapon&.weapon&.element
update_column(:element, new_element) if new_element.present? && self.element != new_element
update_column(:element, new_element) if new_element.present? && element != new_element
end
##
@ -449,7 +450,7 @@ class Party < ApplicationRecord
# @return [void]
def update_extra!
new_extra = weapons.any? { |gw| GridWeapon::EXTRA_POSITIONS.include?(gw.position.to_i) }
update_column(:extra, new_extra) if self.extra != new_extra
update_column(:extra, new_extra) if extra != new_extra
end
##

251
app/services/dataminer.rb Normal file
View file

@ -0,0 +1,251 @@
# frozen_string_literal: true
class Dataminer
include HTTParty
BOT_UID = '39094985'
GAME_VERSION = '1741068713'
base_uri 'https://game.granbluefantasy.jp'
format :json
HEADERS = {
'Accept' => 'application/json, text/javascript, */*; q=0.01',
'Accept-Language' => 'en-US,en;q=0.9',
'Accept-Encoding' => 'gzip, deflate, br, zstd',
'Content-Type' => 'application/json',
'DNT' => '1',
'Origin' => 'https://game.granbluefantasy.jp',
'Referer' => 'https://game.granbluefantasy.jp/',
'User-Agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36',
'X-Requested-With' => 'XMLHttpRequest'
}.freeze
attr_reader :page, :cookies, :logger, :debug
def initialize(page:, access_token:, wing:, midship:, t: 'dummy', debug: false)
@page = page
@cookies = {
access_gbtk: access_token,
wing: wing,
t: t,
midship: midship
}
@debug = debug
setup_logger
end
def fetch
timestamp = Time.now.to_i * 1000
response = self.class.post(
"/#{page}?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}",
headers: HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
)
)
raise AuthenticationError if auth_failed?(response)
response
end
def fetch_character(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/npc_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
style: 1,
character_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Character', granblue_id, response) if response
response
end
def fetch_weapon(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/weapon_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
weapon_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Weapon', granblue_id, response) if response
response
end
def fetch_summon(granblue_id)
timestamp = Time.now.to_i * 1000
url = "/archive/summon_detail?_=#{timestamp}&t=#{timestamp}&uid=#{BOT_UID}"
body = {
special_token: nil,
user_id: BOT_UID,
kind_name: '0',
attribute: '0',
event_id: nil,
story_id: nil,
summon_id: granblue_id
}
response = fetch_detail(url, body)
update_game_data('Summon', granblue_id, response) if response
response
end
# Public batch processing methods
def fetch_all_characters(only_missing: false)
process_all_records('Character', only_missing: only_missing)
end
def fetch_all_weapons(only_missing: false)
process_all_records('Weapon', only_missing: only_missing)
end
def fetch_all_summons(only_missing: false)
process_all_records('Summon', only_missing: only_missing)
end
private
def format_cookies
cookies.map { |k, v| "#{k}=#{v}" }.join('; ')
end
def auth_failed?(response)
return true if response.code != 200
begin
parsed = JSON.parse(response.body)
parsed.is_a?(Hash) && parsed['auth_status'] == 'require_auth'
rescue JSON::ParserError
true
end
end
def setup_logger
@logger = ::Logger.new($stdout)
@logger.level = debug ? ::Logger::DEBUG : ::Logger::INFO
@logger.formatter = proc do |severity, _datetime, _progname, msg|
case severity
when 'DEBUG'
debug ? "#{msg}\n" : ''
else
"#{msg}\n"
end
end
# Suppress SQL logs in non-debug mode
return if debug
ActiveRecord::Base.logger.level = ::Logger::INFO if defined?(ActiveRecord::Base)
end
def fetch_detail(url, body)
logger.debug "\n=== Request Details ==="
logger.debug "URL: #{url}"
logger.debug 'Headers:'
logger.debug HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
).inspect
logger.debug 'Body:'
logger.debug body.to_json
logger.debug '===================='
response = self.class.post(
url,
headers: HEADERS.merge(
'Cookie' => format_cookies,
'X-VERSION' => GAME_VERSION
),
body: body.to_json
)
logger.debug "\n=== Response Details ==="
logger.debug "Response code: #{response.code}"
logger.debug 'Response headers:'
logger.debug response.headers.inspect
logger.debug 'Raw response body:'
logger.debug response.body.inspect
begin
logger.debug 'Parsed response body (if JSON):'
logger.debug JSON.parse(response.body).inspect
rescue JSON::ParserError => e
logger.debug "Could not parse as JSON: #{e.message}"
end
logger.debug '======================'
raise AuthenticationError if auth_failed?(response)
JSON.parse(response.body)
end
def update_game_data(model_name, granblue_id, response_data)
return unless response_data.is_a?(Hash)
model = Object.const_get(model_name)
record = model.find_by(granblue_id: granblue_id)
if record
record.update(game_raw_en: response_data)
logger.debug "Updated #{model_name} #{granblue_id}"
else
logger.warn "#{model_name} with granblue_id #{granblue_id} not found in database"
end
rescue StandardError => e
logger.error "Error updating #{model_name} #{granblue_id}: #{e.message}"
end
def process_all_records(model_name, only_missing: false)
model = Object.const_get(model_name)
scope = model
scope = scope.where(game_raw_en: nil) if only_missing
total = scope.count
success_count = 0
error_count = 0
logger.info "Starting to fetch #{total} #{model_name.downcase}s#{' (missing data only)' if only_missing}..."
scope.find_each do |record|
logger.info "\nProcessing #{model_name} #{record.granblue_id} (#{success_count + error_count + 1}/#{total})"
response = case model_name
when 'Character'
fetch_character(record.granblue_id)
when 'Weapon'
fetch_weapon(record.granblue_id)
when 'Summon'
fetch_summon(record.granblue_id)
end
success_count += 1
logger.debug "Successfully processed #{model_name} #{record.granblue_id}"
sleep(1)
rescue StandardError => e
error_count += 1
logger.error "Error processing #{model_name} #{record.granblue_id}: #{e.message}"
end
logger.info "\nProcessing complete!"
logger.info "Total: #{total}"
logger.info "Successful: #{success_count}"
logger.info "Failed: #{error_count}"
end
class AuthenticationError < StandardError; end
end

View file

@ -25,6 +25,8 @@ common: &default_settings
# agent_enabled: false
log_file_path: logs/
# Logging level for log/newrelic_agent.log
log_level: info

View file

@ -4,20 +4,20 @@
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
max_threads_count = ENV.fetch('RAILS_MAX_THREADS') { 5 }
min_threads_count = ENV.fetch('RAILS_MIN_THREADS') { max_threads_count }
threads min_threads_count, max_threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port ENV.fetch("PORT") { 3000 }
port ENV.fetch('PORT', 3000)
# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch("RAILS_ENV") { "development" }
environment ENV.fetch('RAILS_ENV') { 'development' }
# Specifies the `pidfile` that Puma will use.
pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }
pidfile ENV.fetch('PIDFILE') { 'tmp/pids/server.pid' }
# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together

View file

@ -20,6 +20,9 @@ Rails.application.routes.draw do
get 'version', to: 'api#version'
post 'import', to: 'import#create'
post 'import/weapons', to: 'import#weapons'
post 'import/summons', to: 'import#summons'
post 'import/characters', to: 'import#characters'
get 'users/info/:id', to: 'users#info'

View file

@ -6,119 +6,119 @@ class MigrateWeaponSeries < ActiveRecord::Migration[8.0]
puts 'Starting weapon series migration...'
puts 'Updating Seraphic Weapons (0 -> 1)...'
Weapon.find_by(series: 0).update!(new_series: 1)
Weapon.where(series: 0).update_all(new_series: 1)
puts 'Updating Grand Weapons (1 -> 2)...'
Weapon.find_by(series: 1).update!(new_series: 2)
Weapon.where(series: 1).update_all(new_series: 2)
puts 'Updating Dark Opus Weapons (2 -> 3)...'
Weapon.find_by(series: 2).update!(new_series: 3)
Weapon.where(series: 2).update_all(new_series: 3)
puts 'Updating Revenant Weapons (4 -> 4)...'
Weapon.find_by(series: 4).update!(new_series: 4)
Weapon.where(series: 4).update_all(new_series: 4)
puts 'Updating Primal Weapons (6 -> 5)...'
Weapon.find_by(series: 6).update!(new_series: 5)
Weapon.where(series: 6).update_all(new_series: 5)
puts 'Updating Beast Weapons (5, 7 -> 6)...'
Weapon.find_by(series: 5).update!(new_series: 6)
Weapon.find_by(series: 7).update!(new_series: 6)
Weapon.where(series: 5).update_all(new_series: 6)
Weapon.where(series: 7).update_all(new_series: 6)
puts 'Updating Regalia Weapons (8 -> 7)...'
Weapon.find_by(series: 8).update!(new_series: 7)
Weapon.where(series: 8).update_all(new_series: 7)
puts 'Updating Omega Weapons (9 -> 8)...'
Weapon.find_by(series: 9).update!(new_series: 8)
Weapon.where(series: 9).update_all(new_series: 8)
puts 'Updating Olden Primal Weapons (10 -> 9)...'
Weapon.find_by(series: 10).update!(new_series: 9)
Weapon.where(series: 10).update_all(new_series: 9)
puts 'Updating Hollowsky Weapons (12 -> 10)...'
Weapon.find_by(series: 12).update!(new_series: 10)
Weapon.where(series: 12).update_all(new_series: 10)
puts 'Updating Xeno Weapons (13 -> 11)...'
Weapon.find_by(series: 13).update!(new_series: 11)
Weapon.where(series: 13).update_all(new_series: 11)
puts 'Updating Rose Weapons (15 -> 12)...'
Weapon.find_by(series: 15).update!(new_series: 12)
Weapon.where(series: 15).update_all(new_series: 12)
puts 'Updating Ultima Weapons (17 -> 13)...'
Weapon.find_by(series: 17).update!(new_series: 13)
Weapon.where(series: 17).update_all(new_series: 13)
puts 'Updating Bahamut Weapons (16 -> 14)...'
Weapon.find_by(series: 16).update!(new_series: 14)
Weapon.where(series: 16).update_all(new_series: 14)
puts 'Updating Epic Weapons (18 -> 15)...'
Weapon.find_by(series: 18).update!(new_series: 15)
Weapon.where(series: 18).update_all(new_series: 15)
puts 'Updating Cosmos Weapons (20 -> 16)...'
Weapon.find_by(series: 20).update!(new_series: 16)
Weapon.where(series: 20).update_all(new_series: 16)
puts 'Updating Superlative Weapons (22 -> 17)...'
Weapon.find_by(series: 22).update!(new_series: 17)
Weapon.where(series: 22).update_all(new_series: 17)
puts 'Updating Vintage Weapons (23 -> 18)...'
Weapon.find_by(series: 23).update!(new_series: 18)
Weapon.where(series: 23).update_all(new_series: 18)
puts 'Updating Class Champion Weapons (24 -> 19)...'
Weapon.find_by(series: 24).update!(new_series: 19)
Weapon.where(series: 24).update_all(new_series: 19)
puts 'Updating Sephira Weapons (28 -> 23)...'
Weapon.find_by(series: 28).update!(new_series: 23)
Weapon.where(series: 28).update_all(new_series: 23)
puts 'Updating Astral Weapons (14 -> 26)...'
Weapon.find_by(series: 14).update!(new_series: 26)
Weapon.where(series: 14).update_all(new_series: 26)
puts 'Updating Draconic Weapons (3 -> 27)...'
Weapon.find_by(series: 3).update!(new_series: 27)
Weapon.where(series: 3).update_all(new_series: 27)
puts 'Updating Ancestral Weapons (21 -> 29)...'
Weapon.find_by(series: 21).update!(new_series: 29)
Weapon.where(series: 21).update_all(new_series: 29)
puts 'Updating New World Foundation (29 -> 30)...'
Weapon.find_by(series: 29).update!(new_series: 30)
Weapon.where(series: 29).update_all(new_series: 30)
puts 'Updating Ennead Weapons (19 -> 31)...'
Weapon.find_by(series: 19).update!(new_series: 31)
Weapon.where(series: 19).update_all(new_series: 31)
puts 'Updating Militis Weapons (11 -> 32)...'
Weapon.find_by(series: 11).update!(new_series: 32)
Weapon.where(series: 11).update_all(new_series: 32)
puts 'Updating Malice Weapons (26 -> 33)...'
Weapon.find_by(series: 26).update!(new_series: 33)
Weapon.where(series: 26).update_all(new_series: 33)
puts 'Updating Menace Weapons (26 -> 34)...'
Weapon.find_by(series: 26).update!(new_series: 34)
Weapon.where(series: 26).update_all(new_series: 34)
puts 'Updating Illustrious Weapons (31 -> 35)...'
Weapon.find_by(series: 31).update!(new_series: 35)
Weapon.where(series: 31).update_all(new_series: 35)
puts 'Updating Proven Weapons (25 -> 36)...'
Weapon.find_by(series: 25).update!(new_series: 36)
Weapon.where(series: 25).update_all(new_series: 36)
puts 'Updating Revans Weapons (30 -> 37)...'
Weapon.find_by(series: 30).update!(new_series: 37)
Weapon.where(series: 30).update_all(new_series: 37)
puts 'Updating World Weapons (32 -> 38)...'
Weapon.find_by(series: 32).update!(new_series: 38)
Weapon.where(series: 32).update_all(new_series: 38)
puts 'Updating Exo Weapons (33 -> 39)...'
Weapon.find_by(series: 33).update!(new_series: 39)
Weapon.where(series: 33).update_all(new_series: 39)
puts 'Updating Draconic Weapons Providence (34 -> 40)...'
Weapon.find_by(series: 34).update!(new_series: 40)
Weapon.where(series: 34).update_all(new_series: 40)
puts 'Updating Celestial Weapons (37 -> 41)...'
Weapon.find_by(series: 37).update!(new_series: 41)
Weapon.where(series: 37).update_all(new_series: 41)
puts 'Updating Omega Rebirth Weapons (38 -> 42)...'
Weapon.find_by(series: 38).update!(new_series: 42)
Weapon.where(series: 38).update_all(new_series: 42)
puts 'Updating Event Weapons (34 -> 98)...'
Weapon.find_by(series: 34).update!(new_series: 98) # Event
Weapon.where(series: 34).update_all(new_series: 98) # Event
puts 'Updating Gacha Weapons (36 -> 99)...'
Weapon.find_by(series: 36).update!(new_series: 99) # Gacha
Weapon.where(series: 36).update_all(new_series: 99) # Gacha
puts 'Migration completed successfully!'
rescue StandardError => e

View file

@ -0,0 +1,15 @@
class AddRawDataColumns < ActiveRecord::Migration[8.0]
def change
add_column :characters, :wiki_raw, :text
add_column :characters, :game_raw_en, :text
add_column :characters, :game_raw_jp, :text
add_column :summons, :wiki_raw, :text
add_column :summons, :game_raw_en, :text
add_column :summons, :game_raw_jp, :text
add_column :weapons, :wiki_raw, :text
add_column :weapons, :game_raw_en, :text
add_column :weapons, :game_raw_jp, :text
end
end

View file

@ -0,0 +1,6 @@
class AddClassicIiAndCollabToGacha < ActiveRecord::Migration[8.0]
def change
add_column :gacha, :classic_ii, :boolean, default: false
add_column :gacha, :collab, :boolean, default: false
end
end

View file

@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
ActiveRecord::Schema[8.0].define(version: 2025_03_01_143956) do
# These are extensions that must be enabled in order to support this database
enable_extension "btree_gin"
enable_extension "pg_catalog.plpgsql"
@ -67,6 +67,9 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.string "kamigame", default: ""
t.string "nicknames_en", default: [], null: false, array: true
t.string "nicknames_jp", default: [], null: false, array: true
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_characters_on_granblue_id"
t.index ["name_en"], name: "index_characters_on_name_en", opclass: :gin_trgm_ops, using: :gin
end
@ -420,6 +423,9 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.date "transcendence_date"
t.string "nicknames_en", default: [], null: false, array: true
t.string "nicknames_jp", default: [], null: false, array: true
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_summons_on_granblue_id"
t.index ["name_en"], name: "index_summons_on_name_en", opclass: :gin_trgm_ops, using: :gin
end
@ -495,6 +501,10 @@ ActiveRecord::Schema[8.0].define(version: 2025_02_18_025315) do
t.date "transcendence_date"
t.string "recruits"
t.integer "series"
t.integer "new_series"
t.text "wiki_raw"
t.text "game_raw_en"
t.text "game_raw_jp"
t.index ["granblue_id"], name: "index_weapons_on_granblue_id"
t.index ["name_en"], name: "index_weapons_on_name_en", opclass: :gin_trgm_ops, using: :gin
t.index ["recruits"], name: "index_weapons_on_recruits"

View file

@ -0,0 +1,3 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Yuel (Grand),ユエル(リミテッドver),3040580000,3,2,10,8,2,2,,false,265,1375,,1413,7335,,,,,,false,false,,,{3006},Yuel (Grand),2025-02-27,,,ユエル (SSR)リミテッドバージョン,486749,SSRリミテッドユエル,,
Tsukuyomi,ツクヨミ,3040581000,3,5,6,,2,5,,false,335,1175,,1785,6275,,,,,,false,false,,,{3270},Tsukuyomi,2025-02-27,,,ツクヨミ (SSR),486748,SSRツクヨミ,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Yuel (Grand) ユエル(リミテッドver) 3040580000 3 2 10 8 2 2 false 265 1375 1413 7335 false false {3006} Yuel (Grand) 2025-02-27 ユエル (SSR)リミテッドバージョン 486749 SSRリミテッドユエル
3 Tsukuyomi ツクヨミ 3040581000 3 5 6 2 5 false 335 1175 1785 6275 false false {3270} Tsukuyomi 2025-02-27 ツクヨミ (SSR) 486748 SSRツクヨミ

View file

@ -0,0 +1,7 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
Sennen Goji,千年護持,1040917100,3,2,10,1,true,false,150,15,29,244,,,519,3577,,,false,,false,false,,,2025-02-28,2025-02-28,,Sennen Goji,,486774,千年護持,,,false,,,,,,,,,,,,,,
Nightgaze Gate,夜見之門,1040424200,3,5,6,99,false,false,100,10,46,285,,,369,2163,,,false,,false,false,,,2025-02-28,,,Nightgaze Gate,,,夜見之門,,,false,,,,,,,,,,,,,,
Bane of Avidya,無明滅却杵,1040424100,3,6,6,98,false,false,100,10,34,293,,,227,1744,,,false,,false,false,,,2025-02-26,,,Bane of Avidya,無明滅却杵,486167,無明滅却杵,,,false,,,,,,,,,,,,,,
"
Klesha-Cleansing Dharmachakra",破魔輪宝,1030609800,2,6,7,98,false,false,75,10,22,136,,,197,1272,,,false,,false,false,,,2025-02-26,,,"
Klesha-Cleansing Dharmachakra",破魔輪宝,,破魔輪宝,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 Sennen Goji 千年護持 1040917100 3 2 10 1 true false 150 15 29 244 519 3577 false false false 2025-02-28 2025-02-28 Sennen Goji 486774 千年護持 false
3 Nightgaze Gate 夜見之門 1040424200 3 5 6 99 false false 100 10 46 285 369 2163 false false false 2025-02-28 Nightgaze Gate 夜見之門 false
4 Bane of Avidya 無明滅却杵 1040424100 3 6 6 98 false false 100 10 34 293 227 1744 false false false 2025-02-26 Bane of Avidya 無明滅却杵 486167 無明滅却杵 false
5 Klesha-Cleansing Dharmachakra 破魔輪宝 1030609800 2 6 7 98 false false 75 10 22 136 197 1272 false false false 2025-02-26 Klesha-Cleansing Dharmachakra 破魔輪宝 破魔輪宝 false

View file

@ -0,0 +1,9 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
,,1040619800,,,,,,,,,,,,,,,,,,,,,3040572000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040713900,,,,,,,,,,,,,,,,,,,,,3040573000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040028200,,,,,,,,,,,,,,,,,,,,,3040577000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040121200,,,,,,,,,,,,,,,,,,,,,3040576000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040219700,,,,,,,,,,,,,,,,,,,,,3040579000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040319200,,,,,,,,,,,,,,,,,,,,,3040578000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040917100,,,,,,,,,,,,,,,,,,,,,3040580000,,,,,,,,,,,,,,,,,,,,,,,,,
,,1040424200,,,,,,,,,,,,,,,,,,,,,3040581000,,,,,,,,,,,,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 1040619800 3040572000
3 1040713900 3040573000
4 1040028200 3040577000
5 1040121200 3040576000
6 1040219700 3040579000
7 1040319200 3040578000
8 1040917100 3040580000
9 1040424200 3040581000

View file

@ -0,0 +1,10 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Kaguya (Grand),,3040486000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Kaguya,
,,3040519000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Fenie,
,,3040501000,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Uriel,
,,3040254000,,,,,,,,,,,,,,,,,,,,,,,{2030},,,,,,,,,
,,3040158000,,,,,,,,,,,,,,,,,,,,,,,{3098},,,,,,,,,
,,3040103000,,,,,,,,,,,,,,,,,,,,,,,{3070},,,,,,,,,
,,3040377000,,,,,,,,,,,,,,,,,,,,,,,{3192},,,,,,,,,
,,3040058000,,,,,,,,,,,,,,,,,,,,,,,{3045},,,,,,,,,
,,3040500000,,,,,,,,,,,,,,,,,,,,,,,{1027},,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Kaguya (Grand) 3040486000 Kaguya
3 3040519000 Fenie
4 3040501000 Uriel
5 3040254000 {2030}
6 3040158000 {3098}
7 3040103000 {3070}
8 3040377000 {3192}
9 3040058000 {3045}
10 3040500000 {1027}

View file

@ -0,0 +1,7 @@
name_en,name_jp,granblue_id,rarity,element,series,flb,ulb,max_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,subaura,limit,transcendence,max_atk_xlb,max_hp_xlb,summon_id,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,transcendence_date,nicknames_en,nicknames_jp
,,2040361000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040363000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040368000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040366000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040381000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
,,2040385000,,,,true,,,,,,,,,,,,,,,,,,2025-03-10,,,,,,,,
1 name_en name_jp granblue_id rarity element series flb ulb max_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb subaura limit transcendence max_atk_xlb max_hp_xlb summon_id release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame transcendence_date nicknames_en nicknames_jp
2 2040361000 true 2025-03-10
3 2040363000 true 2025-03-10
4 2040368000 true 2025-03-10
5 2040366000 true 2025-03-10
6 2040381000 true 2025-03-10
7 2040385000 true 2025-03-10

View file

@ -0,0 +1,5 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Basara (Grand),バサラ (リミテッドver),3040582000,3,6,10,7,1,2,,false,192,1128,,1740,8760,,,,,,false,false,,,{3271},Basara,2025-03-17,,,バサラ (SSR)リミテッドバージョン,489323,SSRバサラ,,
Mahira (Summer),マキラ(水着ver),3040584000,3,1,8,7,2,3,,false,281,1217,,1408,8915,,,,,,false,false,,,{3073},Mahira (Summer),2025-03-17,,,マキラ (SSR)水着バージョン,489328,SSR水着マキラ,,
Lu Woh (Summer),ル・オー(水着ver),3040583000,3,4,7,6,0,2,,false,236,1172,,1385,7820,,,,,,false,false,,,{3221},Lu Woh (Summer),2025-03-17,,,ル・オー (SSR)水着バージョン,490119,SSR水着ルオー,,
Joy (Event SSR),ジョイ(イベントSSR),3040588000,3,1,7,5,0,0,,false,108,1080,,1711,9011,,,,,,false,false,,,{2146},Joy (Event SSR),2025-03-11,,,ジョイ (SSR),489593,SSRジョイ,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Basara (Grand) バサラ (リミテッドver) 3040582000 3 6 10 7 1 2 false 192 1128 1740 8760 false false {3271} Basara 2025-03-17 バサラ (SSR)リミテッドバージョン 489323 SSRバサラ
3 Mahira (Summer) マキラ(水着ver) 3040584000 3 1 8 7 2 3 false 281 1217 1408 8915 false false {3073} Mahira (Summer) 2025-03-17 マキラ (SSR)水着バージョン 489328 SSR水着マキラ
4 Lu Woh (Summer) ル・オー(水着ver) 3040583000 3 4 7 6 0 2 false 236 1172 1385 7820 false false {3221} Lu Woh (Summer) 2025-03-17 ル・オー (SSR)水着バージョン 490119 SSR水着ルオー
5 Joy (Event SSR) ジョイ(イベントSSR) 3040588000 3 1 7 5 0 0 false 108 1080 1711 9011 false false {2146} Joy (Event SSR) 2025-03-11 ジョイ (SSR) 489593 SSRジョイ

View file

@ -0,0 +1,8 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
Canifortis,天干地支刀・戌之威,1040917200,3,6,10,2,true,false,150,15,38,215,259,,474,2895,3500,,false,,false,false,3040582000,,2025-03-17,2025-03-17,,Canifortis,天干地支刀・戌之威 (SSR),490121,天干地支刀・戌之威,,,false,,,,,,,,,,,,,,
Tenth Crow of the Clutch,第十酉行筒,1040517200,3,1,9,99,false,false,100,10,33,196,,,434,2606,,,false,,false,false,3040584000,,2025-03-17,,,Tenth Crow of the Clutch,第十酉行筒 (SSR),490124,第十酉行筒,,,false,,,,,,,,,,,,,,
Lu Woh Float,ル・オー・フロート,1040817100,3,4,8,99,false,false,100,10,51,303,,,344,2073,,,false,,false,false,3040583000,,2025-03-17,,,Lu Woh Float,ル・オー・フロート (SSR),490125,ル・オー・フロート,,,false,,,,,,,,,,,,,,
Exo Heliocentrum,神杖エクス・ヘリオセント,1040424300,3,6,6,39,true,false,150,15,41,260,315,,333,2029,2453,,false,,false,false,,,2025-03-22,2025-03-22,,Exo Heliocentrum,神杖エクス・ヘリオセント (SSR),490430,神杖エクス・ヘリオセント,,,false,,,,,,,,,,,,,,
Onmyoji's Reito,陰陽之霊刀,1040917300,3,0,10,19,true,true,200,20,35,,,281,430,,,3856,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Onmyoji%27s_Reito,陰陽之霊刀 (SSR),490757,陰陽之霊刀,,,false,,,,,,,,,,,,,,
Ouranosphaira Ravdos,ウラニアスフェラ・ラヴドス,1040424400,3,0,6,19,true,true,200,20,44,,,371,379,,,3390,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Ouranosphaira Ravdos,ウラニアスフェラ・ラヴドス (SSR),490756,ウラニアスフェラ・ラヴドス,,,false,,,,,,,,,,,,,,
Cometa Sica,コメーテス・シーカ,1040121300,3,0,2,19,true,true,200,20,43,,,397,384,,,3251,false,,false,false,,,2025-03-25,2025-03-25,2025-03-25,Cometa Sica,コメーテス・シーカ (SSR),490753,コメーテス・シーカ,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 Canifortis 天干地支刀・戌之威 1040917200 3 6 10 2 true false 150 15 38 215 259 474 2895 3500 false false false 3040582000 2025-03-17 2025-03-17 Canifortis 天干地支刀・戌之威 (SSR) 490121 天干地支刀・戌之威 false
3 Tenth Crow of the Clutch 第十酉行筒 1040517200 3 1 9 99 false false 100 10 33 196 434 2606 false false false 3040584000 2025-03-17 Tenth Crow of the Clutch 第十酉行筒 (SSR) 490124 第十酉行筒 false
4 Lu Woh Float ル・オー・フロート 1040817100 3 4 8 99 false false 100 10 51 303 344 2073 false false false 3040583000 2025-03-17 Lu Woh Float ル・オー・フロート (SSR) 490125 ル・オー・フロート false
5 Exo Heliocentrum 神杖エクス・ヘリオセント 1040424300 3 6 6 39 true false 150 15 41 260 315 333 2029 2453 false false false 2025-03-22 2025-03-22 Exo Heliocentrum 神杖エクス・ヘリオセント (SSR) 490430 神杖エクス・ヘリオセント false
6 Onmyoji's Reito 陰陽之霊刀 1040917300 3 0 10 19 true true 200 20 35 281 430 3856 false false false 2025-03-25 2025-03-25 2025-03-25 Onmyoji%27s_Reito 陰陽之霊刀 (SSR) 490757 陰陽之霊刀 false
7 Ouranosphaira Ravdos ウラニアスフェラ・ラヴドス 1040424400 3 0 6 19 true true 200 20 44 371 379 3390 false false false 2025-03-25 2025-03-25 2025-03-25 Ouranosphaira Ravdos ウラニアスフェラ・ラヴドス (SSR) 490756 ウラニアスフェラ・ラヴドス false
8 Cometa Sica コメーテス・シーカ 1040121300 3 0 2 19 true true 200 20 43 397 384 3251 false false false 2025-03-25 2025-03-25 2025-03-25 Cometa Sica コメーテス・シーカ (SSR) 490753 コメーテス・シーカ false

View file

@ -0,0 +1,16 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
,,1040022600,,,,,true,,150,15,,,221,,,,2603,,,,,,,,,2025-03-14,,,,,,,,,,,,,,,,,,,,,,
,,1040216800,,,,,true,,150,15,,,264,,,,2380,,,,,,,,,2025-03-14,,,,,,,,,,,,,,,,,,,,,,
,,1040618900,,,,,true,,150,15,,,297,,,,2854,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040713700,,,,,true,,150,15,,,270,,,,3048,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040423400,,,,,true,,150,15,,,354,,,,2568,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040916600,,,,,true,,150,15,,,247,,,,3125,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040119100,,,,,true,,150,15,,,278,,,,2943,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040518000,,,,,true,,150,15,,,213,,,,3267,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040617200,,,,,true,,150,15,,,302,,,,2828,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040916400,,,,,true,,150,15,,,214,,,,3288,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040119600,,,,,true,,150,15,,,291,,,,2879,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040816900,,,,,true,,150,15,,,335,,,,2659,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040026000,,,,,true,,150,15,,,302,,,,2821,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
,,1040816800,,,,,true,,150,15,,,366,,,,2507,,,,,,,,,2025-03-10,,,,,,,,,,,,,,,,,,,,,,
Decorus Sicarius,,1040121100,,,,,,,,,,,,,,,,,,,,,,,,,,Decorus Sicarius,,,,,,,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 1040022600 true 150 15 221 2603 2025-03-14
3 1040216800 true 150 15 264 2380 2025-03-14
4 1040618900 true 150 15 297 2854 2025-03-10
5 1040713700 true 150 15 270 3048 2025-03-10
6 1040423400 true 150 15 354 2568 2025-03-10
7 1040916600 true 150 15 247 3125 2025-03-10
8 1040119100 true 150 15 278 2943 2025-03-10
9 1040518000 true 150 15 213 3267 2025-03-10
10 1040617200 true 150 15 302 2828 2025-03-10
11 1040916400 true 150 15 214 3288 2025-03-10
12 1040119600 true 150 15 291 2879 2025-03-10
13 1040816900 true 150 15 335 2659 2025-03-10
14 1040026000 true 150 15 302 2821 2025-03-10
15 1040816800 true 150 15 366 2507 2025-03-10
16 Decorus Sicarius 1040121100 Decorus Sicarius

View file

@ -0,0 +1,4 @@
name_en,name_jp,granblue_id,rarity,element,proficiency1,proficiency2,gender,race1,race2,flb,min_hp,max_hp,max_hp_flb,min_atk,max_atk,max_atk_flb,base_da,base_ta,ougi_ratio,ougi_ratio_flb,special,ulb,max_hp_ulb,max_atk_ulb,character_id,wiki_en,release_date,flb_date,ulb_date,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp
Seofon (Yukata),シエテ(浴衣ver),3040586000,3,5,1,10,1,1,,false,277,1277,,1777,9777,,,,,,false,false,,,{4007},Seofon (Yukata),2025-03-30,,,シエテ (SSR)浴衣バージョン,489325,SSR浴衣シエテ,,
Vikala (Yukata),ビカラ(浴衣ver),3040585000,3,6,3,7,2,1,,false,280,1550,,1600,8250,,,,,,false,false,,,{3150},Vikala (Yukata),2025-03-30,,,ビカラ (SSR)浴衣バージョン,491855,SSR浴衣ビカラ,,
,,3040087000,,,,,,,,true,300,1600,1900,1500,8000,9500,,,,,,,,,,,2016-06-30,2025-03-29,,ロザミア (SSR),33985,SSRロザミア,,
1 name_en name_jp granblue_id rarity element proficiency1 proficiency2 gender race1 race2 flb min_hp max_hp max_hp_flb min_atk max_atk max_atk_flb base_da base_ta ougi_ratio ougi_ratio_flb special ulb max_hp_ulb max_atk_ulb character_id wiki_en release_date flb_date ulb_date wiki_ja gamewith kamigame nicknames_en nicknames_jp
2 Seofon (Yukata) シエテ(浴衣ver) 3040586000 3 5 1 10 1 1 false 277 1277 1777 9777 false false {4007} Seofon (Yukata) 2025-03-30 シエテ (SSR)浴衣バージョン 489325 SSR浴衣シエテ
3 Vikala (Yukata) ビカラ(浴衣ver) 3040585000 3 6 3 7 2 1 false 280 1550 1600 8250 false false {3150} Vikala (Yukata) 2025-03-30 ビカラ (SSR)浴衣バージョン 491855 SSR浴衣ビカラ
4 3040087000 true 300 1600 1900 1500 8000 9500 2016-06-30 2025-03-29 ロザミア (SSR) 33985 SSRロザミア

View file

@ -0,0 +1,5 @@
name_en,name_jp,granblue_id,rarity,element,proficiency,series,flb,ulb,max_level,max_skill_level,min_hp,max_hp,max_hp_flb,max_hp_ulb,min_atk,max_atk,max_atk_flb,max_atk_ulb,extra,ax_type,limit,ax,recruits,max_awakening_level,release_date,flb_date,ulb_date,wiki_en,wiki_ja,gamewith,kamigame,nicknames_en,nicknames_jp,transcendence,transcendence_date, , , , , , , , , , , , ,
First Fling of the Mischief,第一子行弾弓,1040714000,3,6,5,99,true,false,150,15,46,236,284,,377,2460,2981,,false,,false,false,,,2025-03-30,2025-03-30,,First Fling of the Mischief,第一子行弾弓 (SSR),,第一子行弾弓,,,false,,,,,,,,,,,,,,
Prismatic Trientalis,七彩華刀,1040917900,3,5,10,99,false,false,100,10,18,174,,,515,2737,,,false,,false,false,,,2025-03-30,,,Prismatic Trientalis,七彩華刀 (SSR),,七彩華刀,,,false,,,,,,,,,,,,,,
Scarface,スカーフェイス,1040517300,3,5,9,98,false,false,100,10,11,132,,,408,2221,,,false,,false,false,,,2025-03-29,,,Scarface,スカーフェイス (SSR),491802,スカーフェイス,,,false,,,,,,,,,,,,,,
Henchman,ヘンチマン,1030109100,2,5,2,98,false,false,75,10,8,117,,,271,1368,,,false,,false,false,,,2025-03-29,,,Henchman,ヘンチマン (SR),,ヘンチマン,,,false,,,,,,,,,,,,,,
1 name_en name_jp granblue_id rarity element proficiency series flb ulb max_level max_skill_level min_hp max_hp max_hp_flb max_hp_ulb min_atk max_atk max_atk_flb max_atk_ulb extra ax_type limit ax recruits max_awakening_level release_date flb_date ulb_date wiki_en wiki_ja gamewith kamigame nicknames_en nicknames_jp transcendence transcendence_date
2 First Fling of the Mischief 第一子行弾弓 1040714000 3 6 5 99 true false 150 15 46 236 284 377 2460 2981 false false false 2025-03-30 2025-03-30 First Fling of the Mischief 第一子行弾弓 (SSR) 第一子行弾弓 false
3 Prismatic Trientalis 七彩華刀 1040917900 3 5 10 99 false false 100 10 18 174 515 2737 false false false 2025-03-30 Prismatic Trientalis 七彩華刀 (SSR) 七彩華刀 false
4 Scarface スカーフェイス 1040517300 3 5 9 98 false false 100 10 11 132 408 2221 false false false 2025-03-29 Scarface スカーフェイス (SSR) 491802 スカーフェイス false
5 Henchman ヘンチマン 1030109100 2 5 2 98 false false 75 10 8 117 271 1368 false false false 2025-03-29 Henchman ヘンチマン (SR) ヘンチマン false

View file

@ -35,26 +35,31 @@ module Granblue
# @param verbose [Boolean] When true, enables detailed logging
# @param storage [Symbol] Storage mode (:local, :s3, or :both)
# @return [void]
def initialize(id, test_mode: false, verbose: false, storage: :both)
def initialize(id, test_mode: false, verbose: false, storage: :both, logger: nil)
@id = id
@base_url = base_url
@test_mode = test_mode
@verbose = verbose
@storage = storage
@logger = logger || Logger.new($stdout) # fallback logger
@aws_service = AwsService.new
ensure_directories_exist unless @test_mode
end
# Download images for all sizes
# @param selected_size [String] The size to download
# @return [void]
def download
log_info "-> #{@id}"
def download(selected_size = nil)
log_info("-> #{@id}")
return if @test_mode
SIZES.each_with_index do |size, index|
# If a specific size is provided, use only that; otherwise, use all available sizes.
sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size)
url = build_url(size)
process_download(url, size, path, last: index == SIZES.size - 1)
process_download(url, size, path, last: index == sizes.size - 1)
end
end
@ -128,9 +133,9 @@ module Granblue
download.rewind
# Upload to S3 if it doesn't exist
unless @aws_service.file_exists?(s3_key)
@aws_service.upload_stream(download, s3_key)
end
return if @aws_service.file_exists?(s3_key)
@aws_service.upload_stream(download, s3_key)
end
# Check if file should be downloaded based on storage mode
@ -182,7 +187,7 @@ module Granblue
# Log informational message if verbose
# @param message [String] Message
def log_info(message)
puts message if @verbose
@logger.info(message) if @verbose
end
# Download elemental variant image
@ -197,12 +202,10 @@ module Granblue
filepath = "#{path}/#{filename}"
URI.open(url) do |file|
content = file.read
if content
File.open(filepath, 'wb') do |output|
output.write(content)
end
else
raise "Failed to read content from #{url}"
raise "Failed to read content from #{url}" unless content
File.open(filepath, 'wb') do |output|
output.write(content)
end
end
log_info "-> #{size}:\t#{url}..."

View file

@ -15,24 +15,27 @@ module Granblue
# Downloads images for all variants of a character based on their uncap status.
# Overrides {BaseDownloader#download} to handle character-specific variants.
#
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Skips download if character is not found in database
# @note Downloads FLB/ULB variants only if character has those uncaps
# @see #download_variants
def download
def download(selected_size = nil)
character = Character.find_by(granblue_id: @id)
return unless character
download_variants(character)
download_variants(character, selected_size)
end
private
# Downloads all variants of a character's images
#
# @param character [Character] Character model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Only downloads variants that should exist based on character uncap status
def download_variants(character)
def download_variants(character, selected_size = nil)
# All characters have 01 and 02 variants
variants = %W[#{@id}_01 #{@id}_02]
@ -45,18 +48,22 @@ module Granblue
log_info "Downloading character variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id|
download_variant(variant_id)
download_variant(variant_id, selected_size)
end
end
# Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Character variant ID (e.g., "3040001000_01")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
def download_variant(variant_id)
def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose
return if @test_mode
SIZES.each_with_index do |size, index|
sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size)
url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1)
@ -64,12 +71,18 @@ module Granblue
end
# Builds URL for a specific variant and size
#
# @param variant_id [String] Character variant ID
# @param size [String] Image size variant ("main", "grid", or "square")
# @param size [String] Image size variant ("main", "grid", "square", or "detail")
# @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size)
directory = directory_for_size(size)
"#{@base_url}/#{directory}/#{variant_id}.jpg"
if size == 'detail'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg"
end
end
# Gets object type for file paths and storage keys
@ -85,6 +98,7 @@ module Granblue
end
# Gets directory name for a size variant
#
# @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "f", "grid" -> "m", "square" -> "s"
@ -93,6 +107,7 @@ module Granblue
when 'main' then 'f'
when 'grid' then 'm'
when 'square' then 's'
when 'detail' then 'detail'
end
end
end

View file

@ -15,25 +15,28 @@ module Granblue
# Downloads images for all variants of a summon based on their uncap status.
# Overrides {BaseDownloader#download} to handle summon-specific variants.
#
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Skips download if summon is not found in database
# @note Downloads ULB and transcendence variants only if summon has those uncaps
# @see #download_variants
def download
def download(selected_size = nil)
summon = Summon.find_by(granblue_id: @id)
return unless summon
download_variants(summon)
download_variants(summon, selected_size)
end
private
# Downloads all variants of a summon's images
#
# @param summon [Summon] Summon model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Only downloads variants that should exist based on summon uncap status
# @note Handles special transcendence art variants for 6★ summons
def download_variants(summon)
def download_variants(summon, selected_size = nil)
# All summons have base variant
variants = [@id]
@ -41,26 +44,28 @@ module Granblue
variants << "#{@id}_02" if summon.ulb
# Add Transcendence variants if available
if summon.transcendence
variants.push("#{@id}_03", "#{@id}_04")
end
variants.push("#{@id}_03", "#{@id}_04") if summon.transcendence
log_info "Downloading summon variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id|
download_variant(variant_id)
download_variant(variant_id, selected_size)
end
end
# Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Summon variant ID (e.g., "2040001000_02")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Downloads all size variants (main/grid/square) for the given variant
def download_variant(variant_id)
def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose
return if @test_mode
SIZES.each_with_index do |size, index|
sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size)
url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1)
@ -68,12 +73,17 @@ module Granblue
end
# Builds URL for a specific variant and size
#
# @param variant_id [String] Summon variant ID
# @param size [String] Image size variant ("main", "grid", or "square")
# @param size [String] Image size variant ("main", "grid", "square", or "detail")
# @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size)
directory = directory_for_size(size)
"#{@base_url}/#{directory}/#{variant_id}.jpg"
if size == 'detail'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg"
end
end
# Gets object type for file paths and storage keys
@ -89,14 +99,16 @@ module Granblue
end
# Gets directory name for a size variant
#
# @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "party_main", "grid" -> "party_sub", "square" -> "s"
def directory_for_size(size)
case size.to_s
when 'main' then 'party_main'
when 'grid' then 'party_sub'
when 'main' then 'ls'
when 'grid' then 'm'
when 'square' then 's'
when 'detail' then 'detail'
end
end
end

View file

@ -16,49 +16,54 @@ module Granblue
# Downloads images for all variants of a weapon based on their uncap status.
# Overrides {BaseDownloader#download} to handle weapon-specific variants.
#
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Skips download if weapon is not found in database
# @note Downloads transcendence variants only if weapon has those uncaps
# @see #download_variants
def download
def download(selected_size = nil)
weapon = Weapon.find_by(granblue_id: @id)
return unless weapon
download_variants(weapon)
download_variants(weapon, selected_size)
end
private
# Downloads all variants of a weapon's images
#
# @param weapon [Weapon] Weapon model instance to download images for
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Only downloads variants that should exist based on weapon uncap status
# @note Handles special transcendence art variants for transcendable weapons
def download_variants(weapon)
def download_variants(weapon, selected_size = nil)
# All weapons have base variant
variants = [@id]
# Add transcendence variants if available
if weapon.transcendence
variants.push("#{@id}_02", "#{@id}_03")
end
variants.push("#{@id}_02", "#{@id}_03") if weapon.transcendence
log_info "Downloading weapon variants: #{variants.join(', ')}" if @verbose
variants.each do |variant_id|
download_variant(variant_id)
download_variant(variant_id, selected_size)
end
end
# Downloads a specific variant's images in all sizes
#
# @param variant_id [String] Weapon variant ID (e.g., "1040001000_02")
# @param selected_size [String] The size to download. If nil, downloads all sizes.
# @return [void]
# @note Downloads all size variants (main/grid/square) for the given variant
def download_variant(variant_id)
def download_variant(variant_id, selected_size = nil)
log_info "-> #{variant_id}" if @verbose
return if @test_mode
SIZES.each_with_index do |size, index|
sizes = selected_size ? [selected_size] : SIZES
sizes.each_with_index do |size, index|
path = download_path(size)
url = build_variant_url(variant_id, size)
process_download(url, size, path, last: index == SIZES.size - 1)
@ -66,12 +71,17 @@ module Granblue
end
# Builds URL for a specific variant and size
#
# @param variant_id [String] Weapon variant ID
# @param size [String] Image size variant ("main", "grid", or "square")
# @param size [String] Image size variant ("main", "grid", "square", or "raw")
# @return [String] Complete URL for downloading the image
def build_variant_url(variant_id, size)
directory = directory_for_size(size)
"#{@base_url}/#{directory}/#{variant_id}.jpg"
if size == 'raw'
"#{@base_url}/#{directory}/#{variant_id}.png"
else
"#{@base_url}/#{directory}/#{variant_id}.jpg"
end
end
# Gets object type for file paths and storage keys
@ -87,6 +97,7 @@ module Granblue
end
# Gets directory name for a size variant
#
# @param size [String] Image size variant
# @return [String] Directory name in game asset URL structure
# @note Maps "main" -> "ls", "grid" -> "m", "square" -> "s"
@ -95,6 +106,7 @@ module Granblue
when 'main' then 'ls'
when 'grid' then 'm'
when 'square' then 's'
when 'raw' then 'b'
end
end
end

View file

@ -4,21 +4,26 @@ require 'pry'
module Granblue
module Parsers
# CharacterParser parses character data from gbf.wiki
class CharacterParser
attr_reader :granblue_id
def initialize(granblue_id: String, debug: false)
def initialize(granblue_id: String, debug: false, use_local: false)
@character = Character.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new
@wiki = Granblue::Parsers::Wiki.new
@debug = debug || false
@use_local = use_local
end
# Fetches using @wiki and then processes the response
# Returns true if successful, false if not
# Raises an exception if something went wrong
def fetch(save: false)
if @use_local && @character.wiki_raw.present?
wikitext = @character.wiki_raw
return handle_fetch_success(wikitext, save)
end
response = fetch_wiki_info
return false if response.nil?
@ -49,6 +54,9 @@ module Granblue
# Handle the response from the wiki if the response is successful
# If the save flag is set, it will persist the data to the database
def handle_fetch_success(response, save)
@character.wiki_raw = response
@character.save!
ap "#{@character.granblue_id}: Successfully fetched info for #{@character.wiki_en}" if @debug
extracted = parse_string(response)
info = parse(extracted)
@ -152,12 +160,12 @@ module Granblue
info[:id] = hash['id']
info[:charid] = hash['charid'].scan(/\b\d{4}\b/)
info[:flb] = GranblueWiki.boolean.fetch(hash['5star'], false)
info[:flb] = Granblue::Parsers::Wiki.boolean.fetch(hash['5star'], false)
info[:ulb] = hash['max_evo'].to_i == 6
info[:rarity] = GranblueWiki.rarities.fetch(hash['rarity'], 0)
info[:element] = GranblueWiki.elements.fetch(hash['element'], 0)
info[:gender] = GranblueWiki.genders.fetch(hash['gender'], 0)
info[:rarity] = Granblue::Parsers::Wiki.rarities.fetch(hash['rarity'], 0)
info[:element] = Granblue::Parsers::Wiki.elements.fetch(hash['element'], 0)
info[:gender] = Granblue::Parsers::Wiki.genders.fetch(hash['gender'], 0)
info[:proficiencies] = proficiencies_from_hash(hash['weapon'])
info[:races] = races_from_hash(hash['race'])
@ -211,14 +219,14 @@ module Granblue
# Converts proficiencies from a string to a hash
def proficiencies_from_hash(character)
character.to_s.split(',').map.with_index do |prof, i|
{ "proficiency#{i + 1}" => GranblueWiki.proficiencies[prof] }
{ "proficiency#{i + 1}" => Granblue::Parsers::Wiki.proficiencies[prof] }
end.reduce({}, :merge)
end
# Converts races from a string to a hash
def races_from_hash(race)
race.to_s.split(',').map.with_index do |r, i|
{ "race#{i + 1}" => GranblueWiki.races[r] }
{ "race#{i + 1}" => Granblue::Parsers::Wiki.races[r] }
end.reduce({}, :merge)
end

View file

@ -10,7 +10,7 @@ module Granblue
def initialize(granblue_id: String, debug: false)
@summon = Summon.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new(debug: debug)
@wiki = Granblue::Parsers::Wiki.new
@debug = debug || false
end

View file

@ -10,7 +10,7 @@ module Granblue
def initialize(granblue_id: String, debug: false)
@weapon = Weapon.find_by(granblue_id: granblue_id)
@wiki = GranblueWiki.new(debug: debug)
@wiki = Granblue::Parsers::Wiki.new
@debug = debug || false
end
@ -278,17 +278,17 @@ module Granblue
# Converts rarities from a string to a hash
def rarity_from_hash(string)
string ? GranblueWiki.rarities[string.upcase] : nil
string ? Granblue::Parsers::Wiki.rarities[string.upcase] : nil
end
# Converts proficiencies from a string to a hash
def proficiency_from_hash(string)
GranblueWiki.proficiencies[string]
Granblue::Parsers::Wiki.proficiencies[string]
end
# Converts a bullet type from a string to a hash
def bullet_from_hash(string)
string ? GranblueWiki.bullets[string] : nil
string ? Granblue::Parsers::Wiki.bullets[string] : nil
end
# Parses a date string into a Date object

42
lib/tasks/database.rake Normal file
View file

@ -0,0 +1,42 @@
namespace :db do
desc 'Backup remote PostgreSQL database'
task :backup do
remote_host = ENV.fetch('REMOTE_DB_HOST', 'roundhouse.proxy.rlwy.net')
remote_port = ENV.fetch('REMOTE_DB_PORT', '54629')
remote_user = ENV.fetch('REMOTE_DB_USER', 'postgres')
remote_db = ENV.fetch('REMOTE_DB_NAME', 'railway')
password = ENV.fetch('REMOTE_DB_PASSWORD') { raise 'Please set REMOTE_DB_PASSWORD' }
backup_dir = File.expand_path('backups')
FileUtils.mkdir_p(backup_dir)
backup_file = File.join(backup_dir, "#{Time.now.strftime('%Y%m%d_%H%M%S')}-prod-backup.tar")
cmd = %W[
pg_dump -h #{remote_host} -p #{remote_port} -U #{remote_user} -d #{remote_db} -F t
--no-owner --exclude-extension=timescaledb --exclude-extension=timescaledb_toolkit
].join(' ')
puts "Backing up remote database to #{backup_file}..."
system({ 'PGPASSWORD' => password }, "#{cmd} > #{backup_file}")
puts 'Backup completed!'
end
desc 'Restore PostgreSQL database from backup'
task :restore, [:backup_file] => [:environment] do |_, args|
local_user = ENV.fetch('LOCAL_DB_USER', 'justin')
local_db = ENV.fetch('LOCAL_DB_NAME', 'hensei_dev')
# Use the specified backup file or find the most recent one
backup_dir = File.expand_path('backups')
backup_file = args[:backup_file] || Dir.glob("#{backup_dir}/*-prod-backup.tar").max
raise 'Backup file not found. Please specify a valid backup file.' unless backup_file && File.exist?(backup_file)
puts "Restoring database from #{backup_file}..."
system("pg_restore --no-owner --role=#{local_user} --disable-triggers -U #{local_user} -d #{local_db} #{backup_file}")
puts 'Restore completed!'
end
desc 'Backup remote database and restore locally'
task backup_and_restore: %i[backup restore]
end

View file

@ -1,40 +1,50 @@
namespace :granblue do
def _progress_reporter(count:, total:, result:, bar_len: 40, multi: true)
filled_len = (bar_len * count / total).round
status = File.basename(result)
percents = (100.0 * count / total).round(1)
bar = '=' * filled_len + '-' * (bar_len - filled_len)
desc 'Downloads all images for the given object type'
# Downloads all images for a specific type of game object (e.g. summons, weapons)
# Uses the appropriate downloader class based on the object type
#
# @param object [String] Type of object to download images for (e.g. 'summon', 'weapon')
# @example Download all summon images
# rake granblue:download_all_images\[summon\]
# @example Download all weapon images
# rake granblue:download_all_images\[weapon\]
# @example Download all character images
# rake granblue:download_all_images\[character\]
task :download_all_images, %i[object threads size] => :environment do |_t, args|
require 'parallel'
require 'logger'
if !multi
print("[#{bar}] #{percents}% ...#{' ' * 14}#{status}\n")
else
print "\n"
end
end
# Use a thread-safe logger (or Rails.logger if preferred)
logger = Logger.new($stdout)
logger.level = Logger::INFO # set to WARN or INFO to reduce debug noise
desc 'Downloads images for the given object type at the given size'
task :download_all_images, %i[object size] => :environment do |_t, args|
require 'open-uri'
# Load downloader classes
require_relative '../granblue/downloaders/base_downloader'
Dir[Rails.root.join('lib', 'granblue', 'downloaders', '*.rb')].each { |file| require file }
filename = "export/#{args[:object]}-#{args[:size]}.txt"
count = `wc -l #{filename}`.split.first.to_i
object = args[:object]
specified_size = args[:size]
klass = object.classify.constantize
ids = klass.pluck(:granblue_id)
path = "#{Rails.root}/download/#{args[:object]}-#{args[:size]}"
FileUtils.mkdir_p(path) unless Dir.exist?(path)
puts "Downloading images for #{ids.count} #{object.pluralize}..."
puts "Downloading #{count} images from #{args[:object]}-#{args[:size]}.txt..."
if File.exist?(filename)
File.readlines(filename).each_with_index do |line, i|
download = URI.parse(line.strip).open
download_URI = "#{path}/#{download.base_uri.to_s.split('/')[-1]}"
if File.exist?(download_URI)
puts "Skipping #{line}"
logger.info "Downloading images for #{ids.count} #{object.pluralize}..."
thread_count = (args[:threads] || 4).to_i
logger.info "Using #{thread_count} threads for parallel downloads..."
logger.info "Downloading only size: #{specified_size}" if specified_size
Parallel.each(ids, in_threads: thread_count) do |id|
ActiveRecord::Base.connection_pool.with_connection do
downloader_class = "Granblue::Downloaders::#{object.classify}Downloader".constantize
downloader = downloader_class.new(id, verbose: true, logger: logger)
if specified_size
downloader.download(specified_size)
else
IO.copy_stream(download, "#{path}/#{download.base_uri.to_s.split('/')[-1]}")
_progress_reporter(count: i, total: count, result: download_URI, bar_len: 40, multi: false)
downloader.download
end
rescue StandardError => e
puts "#{e}: #{line}"
logger.error "Error downloading #{object} #{id}: #{e.message}"
end
end
end

79
lib/tasks/fetch_wiki.rake Normal file
View file

@ -0,0 +1,79 @@
namespace :granblue do
desc <<~DESC
Fetch and store raw wiki data for objects (Character, Weapon, Summon).
Usage:
rake granblue:fetch_wiki_data # Fetch all Characters (default)
rake granblue:fetch_wiki_data type=Weapon # Fetch all Weapons
rake granblue:fetch_wiki_data type=Summon # Fetch all Summons
rake granblue:fetch_wiki_data type=Character id=5 # Fetch specific Character by ID
rake granblue:fetch_wiki_data force=true # Force re-download even if data exists
DESC
task fetch_wiki_data: :environment do
# Get parameters from environment
type = (ENV['type'] || 'Character').classify
id = ENV['id']
force = ENV['force'] == 'true'
# Validate object type
valid_types = %w[Character Weapon Summon]
unless valid_types.include?(type)
puts "Error: Invalid type '#{type}'. Must be one of: #{valid_types.join(', ')}"
exit 1
end
# Get the class from the type string
klass = type.constantize
# Setup query - either all objects or specific one
query = id.present? ? klass.where(granblue_id: id) : klass.all
errors = []
count = 0
query.find_each do |object|
# Skip objects that already have wiki_raw if force is not set
if object.wiki_raw.present? && !force
puts "Skipping #{object.name_en} (already has wiki_raw)."
next
end
# If the object doesn't have a wiki page specified, skip
if object.wiki_en.blank?
puts "Skipping #{object.name_en} (no wiki_en set)."
next
end
begin
# 1) Fetch raw wikitext from the wiki
wiki_text = Granblue::Parsers::Wiki.new.fetch(object.wiki_en)
# 2) Check if the page is a redirect
redirect_match = wiki_text.match(/#REDIRECT \[\[(.*?)\]\]/)
if redirect_match
redirect_target = redirect_match[1]
# Update object to new wiki_en so we don't keep fetching the old page
object.update!(wiki_en: redirect_target)
# Fetch again with the new page name
wiki_text = Granblue::Parsers::Wiki.new.fetch(redirect_target)
end
puts wiki_text
# 3) Save raw wiki text in the object record
object.update!(wiki_raw: wiki_text)
puts "Saved wiki data for #{object.name_en} (#{object.id})"
count += 1
rescue StandardError => e
errors << { object_id: object.id, type: type, error: e.message }
puts "Error fetching data for #{object.name_en}: #{e.message}"
end
end
if errors.any?
puts "#{errors.size} #{type.pluralize} had errors:"
errors.each { |err| puts " - #{err[:type]} ##{err[:object_id]} => #{err[:error]}" }
else
puts "Wiki data fetch complete for #{count} #{type.pluralize} with no errors!"
end
end
end

View file

@ -0,0 +1,17 @@
[program:hensei-api]
command=/opt/homebrew/bin/mise run s
process_name=%(program_name)s
numprocs=1
directory=/Users/justin/Developer/Granblue/hensei-api
environment=HOME="/Users/justin",MISE_CONFIG_ROOT="/Users/justin/Developer/Granblue/hensei-api",RAILS_ENV="development"
autostart=true
autorestart=unexpected
stopsignal=TERM
user=justin
stdout_logfile=/Users/justin/Developer/Granblue/hensei-api/logs/hensei-api.stdout.log
stdout_logfile_maxbytes=500KB
stdout_logfile_backups=10
stderr_logfile=/Users/justin/Developer/Granblue/hensei-api/logs/hensei-api.stderr.log
stderr_logfile_maxbytes=500KB
stderr_logfile_backups=10
serverurl=AUTO