* Remove ap call * Fix remix render method * Downcase username on db end There was a bug where users with capital letters in their name could not access their profiles after we tried to make things case insensitive. * Remove ap call and unused code * Add granblue.team to cors This works now! * Implement all-entity search to support tagging objects (#117) * Add table for multisearch * Add new route for searching all entities * Make models multisearchable We're going to start with Character, Summon, Weapon and Jobs * Add method to Search controller This will search with trigram first, and then if there aren't enough results, search with prefixed text search * Add support for Japanese all-entity search * Update grid_summons_controller.rb Set the proper uncap level for transcended summons * Search is broken in Japanese! * Grid model object updates * Adds has_one association to canonical objects * GridWeapon is_mainhand refactored * GridCharacter add_awakening refactored * (WIP) Add support for inclusion/exclusion + refactor This commit adds basic support for including/excluding objects from collection filters. There is also a refactor of the filter logic as a whole. It is only implemented in `teams` for now and is a work in progress. * Revert "Grid model object updates" This reverts commit70e820b781. * Revert "(WIP) Add support for inclusion/exclusion + refactor" This reverts commitb5f9889c00. * Add new dependencies * Update database with new columns * Create WikiError This is modeled after the errors we might receive from the wiki * Update GranblueWiki This is an adaptation and cleanup of the original GranblueWiki class. We extracted the object-related code into a parser, and this class is now only responsible for requests and fetching common property maps. * Create CharacterParser The CharacterParser is responsible for taking the response from a wiki object and turning it into something usable. Most of the logic from the original GranblueWiki object ended up here. To use, you can instantiate a CharacterParser with a Granblue ID. It will create the wiki object for you using the provided character and fetch data when you call `fetch` on the CharacterParser. You can also fetch data for all characters with the static method `fetch_all` Currently a specific subset of fields are persisted, but in the future more can and probably should be saved * Add data tables for weapons and summons This lets us store wiki data and release dates for weapons and summons * Add mapping for bullets * Properly pass props * Update debug string * Update character parser to use correct vars * Add weapon parser The weapon parser parses through weapon pages. It has the ability to parse skills but that is not complete yet. It can go through every weapon on the wiki with the aid of the new `wiki_en` column and parse basic data successfully * Update code for FLB If a weapon has ULB, it has FLB too * Add summon parser The summon parser parses through summon pages. Aura/Call parsing has not been added. * Add XLB date column for summons * Add fetch_from_list static method This backports the `fetch_from_list` static method to CharacterParser and WeaponParser. This will let us fetch data for a specific set of items instead of having to fetch one-by-one or fetch the entire dataset again.
296 lines
8.2 KiB
Ruby
296 lines
8.2 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
require 'pry'
|
|
|
|
# WeaponParser parses weapon data from gbf.wiki
|
|
class WeaponParser
|
|
attr_reader :granblue_id
|
|
|
|
def initialize(granblue_id: String, debug: false)
|
|
@weapon = Weapon.find_by(granblue_id: granblue_id)
|
|
@wiki = GranblueWiki.new(debug: debug)
|
|
@debug = debug || false
|
|
end
|
|
|
|
# Fetches using @wiki and then processes the response
|
|
# Returns true if successful, false if not
|
|
# Raises an exception if something went wrong
|
|
def fetch(save: false)
|
|
response = fetch_wiki_info
|
|
return false if response.nil?
|
|
|
|
# return response if response[:error]
|
|
|
|
handle_fetch_success(response, save)
|
|
end
|
|
|
|
private
|
|
|
|
# Handle the response from the wiki if the response is successful
|
|
# If the save flag is set, it will persist the data to the database
|
|
def handle_fetch_success(response, save)
|
|
ap "#{@weapon.granblue_id}: Successfully fetched info for #{@weapon.wiki_en}" if @debug
|
|
extracted = parse_string(response)
|
|
|
|
unless extracted[:template].nil?
|
|
template = @wiki.fetch("Template:#{extracted[:template]}")
|
|
extracted.merge!(parse_string(template))
|
|
end
|
|
|
|
info, skills = parse(extracted)
|
|
|
|
# ap info
|
|
# ap skills
|
|
|
|
persist(info[:info]) if save
|
|
true
|
|
end
|
|
|
|
# Fetches the wiki info from the wiki
|
|
# Returns the response body
|
|
# Raises an exception if something went wrong
|
|
def fetch_wiki_info
|
|
@wiki.fetch(@weapon.wiki_en)
|
|
rescue WikiError => e
|
|
ap e
|
|
# ap "There was an error fetching #{e.page}: #{e.message}" if @debug
|
|
{
|
|
error: {
|
|
name: @weapon.wiki_en,
|
|
granblue_id: @weapon.granblue_id
|
|
}
|
|
}
|
|
end
|
|
|
|
# Iterates over all weapons in the database and fetches their data
|
|
# If the save flag is set, data is saved to the database
|
|
# If the overwrite flag is set, data is fetched even if it already exists
|
|
# If the debug flag is set, additional information is printed to the console
|
|
def self.fetch_all(save: false, overwrite: false, debug: false, start: nil)
|
|
errors = []
|
|
|
|
weapons = Weapon.all.order(:granblue_id)
|
|
|
|
start_index = start.nil? ? 0 : weapons.index { |w| w.granblue_id == start }
|
|
count = weapons.drop(start_index).count
|
|
|
|
# ap "Start index: #{start_index}"
|
|
|
|
weapons.drop(start_index).each_with_index do |w, i|
|
|
percentage = ((i + 1) / count.to_f * 100).round(2)
|
|
ap "#{percentage}%: Fetching #{w.wiki_en}... (#{i + 1}/#{count})" if debug
|
|
next if w.wiki_en.include?('Element Changed') || w.wiki_en.include?('Awakened')
|
|
next unless w.release_date.nil? || overwrite
|
|
|
|
begin
|
|
WeaponParser.new(granblue_id: w.granblue_id,
|
|
debug: debug).fetch(save: save)
|
|
rescue WikiError => e
|
|
errors.push(e.page)
|
|
end
|
|
end
|
|
|
|
ap 'The following pages were unable to be fetched:'
|
|
ap errors
|
|
end
|
|
|
|
def self.fetch_list(list: [], save: false, overwrite: false, debug: false, start: nil)
|
|
errors = []
|
|
|
|
start_index = start.nil? ? 0 : list.index { |id| id == start }
|
|
count = list.drop(start_index).count
|
|
|
|
# ap "Start index: #{start_index}"
|
|
|
|
list.drop(start_index).each_with_index do |id, i|
|
|
weapon = Weapon.find_by(granblue_id: id)
|
|
percentage = ((i + 1) / count.to_f * 100).round(2)
|
|
ap "#{percentage}%: Fetching #{weapon.wiki_en}... (#{i + 1}/#{count})" if debug
|
|
next unless weapon.release_date.nil? || overwrite
|
|
|
|
begin
|
|
WeaponParser.new(granblue_id: weapon.granblue_id,
|
|
debug: debug).fetch(save: save)
|
|
rescue WikiError => e
|
|
errors.push(e.page)
|
|
end
|
|
end
|
|
|
|
ap 'The following pages were unable to be fetched:'
|
|
ap errors
|
|
end
|
|
|
|
# Parses the response string into a hash
|
|
def parse_string(string)
|
|
data = {}
|
|
lines = string.split("\n")
|
|
stop_loop = false
|
|
|
|
lines.each do |line|
|
|
next if stop_loop
|
|
|
|
if line.include?('Gameplay Notes')
|
|
stop_loop = true
|
|
next
|
|
end
|
|
|
|
if line.starts_with?('{{')
|
|
substr = line[2..].strip! || line[2..]
|
|
|
|
# All template tags start with {{ so we can skip the first two characters
|
|
disallowed = %w[#vardefine #lsth About]
|
|
next if substr.start_with?(*disallowed)
|
|
|
|
if substr.start_with?('Weapon')
|
|
ap "--> Found template: #{substr}" if @debug
|
|
|
|
substr = substr.split('|').first
|
|
data[:template] = substr if substr != 'Weapon'
|
|
next
|
|
end
|
|
end
|
|
|
|
next unless line[0] == '|' && line.size > 2
|
|
|
|
key, value = line[1..].split('=', 2).map(&:strip)
|
|
|
|
regex = /\A\{\{\{.*\|\}\}\}\z/
|
|
next if value =~ regex
|
|
|
|
data[key] = value if value
|
|
end
|
|
|
|
data
|
|
end
|
|
|
|
# Parses the hash into a format that can be saved to the database
|
|
def parse(hash)
|
|
info = {}
|
|
skills = {}
|
|
|
|
info[:name] = { en: hash['name'], ja: hash['jpname'] }
|
|
info[:flavor] = { en: hash['flavor'], ja: hash['jpflavor'] }
|
|
info[:id] = hash['id']
|
|
|
|
info[:flb] = hash['evo_max'].to_i >= 4
|
|
info[:ulb] = hash['evo_max'].to_i == 5
|
|
|
|
info[:rarity] = rarity_from_hash(hash['rarity'])
|
|
info[:proficiency] = proficiency_from_hash(hash['weapon'])
|
|
info[:series] = hash['series']
|
|
info[:obtain] = hash['obtain']
|
|
|
|
if hash.key?('bullets')
|
|
info[:bullets] = {
|
|
count: hash['bullets'].to_i,
|
|
loadout: [
|
|
bullet_from_hash(hash['bullet1']),
|
|
bullet_from_hash(hash['bullet2']),
|
|
bullet_from_hash(hash['bullet3']),
|
|
bullet_from_hash(hash['bullet4']),
|
|
bullet_from_hash(hash['bullet5']),
|
|
bullet_from_hash(hash['bullet6'])
|
|
]
|
|
}
|
|
end
|
|
|
|
info[:hp] = {
|
|
min_hp: hash['hp1'].to_i,
|
|
max_hp: hash['hp2'].to_i,
|
|
max_hp_flb: hash['hp3'].to_i,
|
|
max_hp_ulb: hash['hp4'].to_i.zero? ? nil : hash['hp4'].to_i
|
|
}
|
|
|
|
info[:atk] = {
|
|
min_atk: hash['atk1'].to_i,
|
|
max_atk: hash['atk2'].to_i,
|
|
max_atk_flb: hash['atk3'].to_i,
|
|
max_atk_ulb: hash['atk4'].to_i.zero? ? nil : hash['atk4'].to_i
|
|
}
|
|
|
|
info[:dates] = {
|
|
release_date: parse_date(hash['release_date']),
|
|
flb_date: parse_date(hash['4star_date']),
|
|
ulb_date: parse_date(hash['5star_date'])
|
|
}
|
|
|
|
info[:links] = {
|
|
wiki: { en: hash['name'], ja: hash['link_jpwiki'] },
|
|
gamewith: hash['link_gamewith'],
|
|
kamigame: hash['link_kamigame']
|
|
}
|
|
|
|
skills[:charge_attack] = {
|
|
name: { en: hash['ougi_name'], ja: hash['jpougi_name'] },
|
|
description: {
|
|
mlb: {
|
|
en: hash['enougi'],
|
|
ja: hash['jpougi']
|
|
},
|
|
flb: {
|
|
en: hash['enougi_4s'],
|
|
ja: hash['jpougi_4s']
|
|
}
|
|
}
|
|
}
|
|
|
|
skills[:skills] = [
|
|
{
|
|
name: { en: hash['s1_name'], ja: nil },
|
|
description: { en: hash['ens1_desc'] || hash['s1_desc'], ja: nil }
|
|
},
|
|
{
|
|
name: { en: hash['s2_name'], ja: nil },
|
|
description: { en: hash['ens2_desc'] || hash['s2_desc'], ja: nil }
|
|
},
|
|
{
|
|
name: { en: hash['s3_name'], ja: nil },
|
|
description: { en: hash['ens3_desc'] || hash['s3_desc'], ja: nil }
|
|
}
|
|
]
|
|
|
|
{
|
|
info: info.compact,
|
|
skills: skills.compact
|
|
}
|
|
end
|
|
|
|
# Saves select fields to the database
|
|
def persist(hash)
|
|
@weapon.release_date = hash[:dates][:release_date]
|
|
@weapon.flb_date = hash[:dates][:flb_date] if hash[:dates].key?(:flb_date)
|
|
@weapon.ulb_date = hash[:dates][:ulb_date] if hash[:dates].key?(:ulb_date)
|
|
|
|
@weapon.wiki_ja = hash[:links][:wiki][:ja] if hash[:links].key?(:wiki) && hash[:links][:wiki].key?(:ja)
|
|
@weapon.gamewith = hash[:links][:gamewith] if hash[:links].key?(:gamewith)
|
|
@weapon.kamigame = hash[:links][:kamigame] if hash[:links].key?(:kamigame)
|
|
|
|
if @weapon.save
|
|
ap "#{@weapon.granblue_id}: Successfully saved info for #{@weapon.wiki_en}" if @debug
|
|
puts
|
|
true
|
|
end
|
|
|
|
false
|
|
end
|
|
|
|
# Converts rarities from a string to a hash
|
|
def rarity_from_hash(string)
|
|
string ? GranblueWiki.rarities[string.upcase] : nil
|
|
end
|
|
|
|
# Converts proficiencies from a string to a hash
|
|
def proficiency_from_hash(string)
|
|
GranblueWiki.proficiencies[string]
|
|
end
|
|
|
|
# Converts a bullet type from a string to a hash
|
|
def bullet_from_hash(string)
|
|
string ? GranblueWiki.bullets[string] : nil
|
|
end
|
|
|
|
# Parses a date string into a Date object
|
|
def parse_date(date_str)
|
|
Date.parse(date_str) unless date_str.blank?
|
|
end
|
|
end
|