granblue.team/hensei-api
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
hensei-api/docs/rake-tasks.md

11 KiB
Raw Blame History

Rake Tasks Documentation

Complete reference for all available rake tasks in the hensei-api project. These tasks handle data management, image downloading, wiki synchronization, and various maintenance operations.

Quick Reference

# List all available tasks
rake -T

# List Granblue-specific tasks
rake -T granblue

# List export tasks
rake -T export

# Get detailed help for a task
rake -D granblue:import_data

Image Download Tasks

Character Images

granblue:export:character_images

Download character images with all variants and sizes.

# Download all characters
rake granblue:export:character_images

# Download specific character
rake granblue:export:character_images[3040001000]

# With options: [id,test_mode,verbose,storage,size]
rake granblue:export:character_images[3040001000,false,true,s3,grid]

# Test mode (simulate)
rake granblue:export:character_images[,true,true]

# Download only specific size for all
rake granblue:export:character_images[,false,true,both,detail]

Parameters:

  • id: Granblue ID or empty for all
  • test_mode: true/false (default: false)
  • verbose: true/false (default: true)
  • storage: local/s3/both (default: both)
  • size: main/grid/square/detail (default: all)

Weapon Images

granblue:export:weapon_images

Download weapon images including elemental variants.

# Download all weapons
rake granblue:export:weapon_images

# Download specific weapon
rake granblue:export:weapon_images[1040001000]

# S3 only, grid size
rake granblue:export:weapon_images[1040001000,false,true,s3,grid]

Parameters: Same as character_images

Summon Images

granblue:export:summon_images

Download summon images with uncap variants.

# Download all summons
rake granblue:export:summon_images

# Download specific summon
rake granblue:export:summon_images[2040001000]

# Local storage only
rake granblue:export:summon_images[2040001000,false,true,local]

Parameters: Same as character_images

Job Images

granblue:export:job_images

Download job images with gender variants (wide and zoom formats).

# Download all jobs
rake granblue:export:job_images

# Download specific job
rake granblue:export:job_images[100401]

# Download only zoom images
rake granblue:export:job_images[100401,false,true,both,zoom]

# Download only wide images
rake granblue:export:job_images[,false,true,both,wide]

Parameters: Same as character_images, but size is wide/zoom

Bulk Download

granblue:download_all_images

Download all images for a type with parallel processing.

# Download with parallel threads
rake granblue:download_all_images[character,4]
rake granblue:download_all_images[weapon,8]
rake granblue:download_all_images[summon,4]
rake granblue:download_all_images[job,2]

# Download specific size with threads
rake granblue:download_all_images[character,4,grid]
rake granblue:download_all_images[weapon,8,main]

Parameters:

  • object: character/weapon/summon/job
  • threads: Number of parallel threads (default: 4)
  • size: Specific size to download (optional)

Data Import Tasks

Import from CSV

granblue:import_data

Import character, weapon, and summon data from CSV files.

# Import all CSV files from db/seed/updates/
rake granblue:import_data

# Test mode - validate without importing
rake granblue:import_data TEST=true

# Verbose output
rake granblue:import_data VERBOSE=true

# Both test and verbose
rake granblue:import_data TEST=true VERBOSE=true

CSV Location: db/seed/updates/ File naming: {type}_YYYYMMDD.csv

Wiki Data Tasks

Fetch Wiki Data

granblue:fetch_wiki_data

Fetch and store wiki data for game objects.

# Fetch all characters (default)
rake granblue:fetch_wiki_data

# Fetch specific type
rake granblue:fetch_wiki_data type=Weapon
rake granblue:fetch_wiki_data type=Summon
rake granblue:fetch_wiki_data type=Character

# Fetch specific item
rake granblue:fetch_wiki_data type=Character id=3040001000
rake granblue:fetch_wiki_data type=Weapon id=1040001000

# Force re-fetch even if data exists
rake granblue:fetch_wiki_data force=true
rake granblue:fetch_wiki_data type=Summon force=true

Parameters:

  • type: Character/Weapon/Summon (default: Character)
  • id: Specific Granblue ID (optional)
  • force: true/false - Re-fetch even if wiki_raw exists

Export URL Tasks

Job URLs

granblue:export:job

Export list of job image URLs to text file.

# Export icon URLs
rake granblue:export:job[icon]

# Export portrait URLs
rake granblue:export:job[portrait]

Output: export/job-{size}.txt

Export All URLs

granblue:export:all

Export all asset URLs for batch downloading.

# Export all URL lists
rake granblue:export:all

# Creates files in export/
# - character-*.txt
# - weapon-*.txt
# - summon-*.txt
# - job-*.txt

Preview Tasks

Generate Previews

previews:generate_all

Generate preview images for all parties without previews.

# Generate missing previews
rake previews:generate_all

# Processes parties with pending/failed/nil preview_state
# Uploads to S3: previews/{shortcode}.png

previews:regenerate_all

Regenerate preview images for all parties.

# Regenerate all previews (overwrites existing)
rake previews:regenerate_all

Database Tasks

Database Management

database:backup

Create database backup.

# Create timestamped backup
rake database:backup

# Output: backups/hensei_YYYYMMDD_HHMMSS.sql

database:restore

Restore database from backup.

# Restore from latest backup
rake database:restore

# Restore specific backup
rake database:restore[backups/hensei_20240315_120000.sql]

Deployment Tasks

deploy:post

Run post-deployment tasks.

# Run all post-deployment tasks
rake deploy:post

# Includes:
# - Database migrations
# - Data imports
# - Asset compilation
# - Cache clearing

Utility Tasks

Download Images

granblue:download_images

Legacy task for downloading images.

# Download images (legacy)
rake granblue:download_images

Note: Prefer using the newer export tasks.

Export Accessories

granblue:export:accessories

Export accessory data and images.

# Export all accessories
rake granblue:export:accessories

Task Patterns and Options

Common Parameters

Most tasks support these common parameters:

Parameter Values Description
test_mode true/false Simulate without making changes
verbose true/false Enable detailed logging
storage local/s3/both Storage destination
force true/false Force operation even if data exists

Environment Variables

Tasks can use environment variables:

# Test mode
TEST=true rake granblue:import_data

# Verbose output
VERBOSE=true rake granblue:import_data

# Force re-processing
FORCE=true rake granblue:fetch_wiki_data

# Custom environment
RAILS_ENV=production rake granblue:export:character_images

Parameter Passing

Two ways to pass parameters:

  1. Bracket notation (for defined parameters):
rake task_name[param1,param2,param3]
  1. Environment variables (for options):
TEST=true VERBOSE=true rake task_name

Skipping Parameters

Use commas to skip parameters:

# Skip ID, set test_mode=true, verbose=true
rake granblue:export:character_images[,true,true]

# Skip ID and test_mode, set verbose=true, storage=s3
rake granblue:export:weapon_images[,,true,s3]

Best Practices

1. Test Before Running

Always test operations first:

# Test mode
rake granblue:import_data TEST=true

# Verify output
rake granblue:export:character_images[,true,true]

# Then run for real
rake granblue:export:character_images

2. Use Parallel Processing

For bulk operations, use parallel threads:

# Good - parallel
rake granblue:download_all_images[character,8]

# Less efficient - sequential
rake granblue:export:character_images

3. Monitor Long-Running Tasks

Use verbose mode and logs:

# Enable verbose
rake granblue:export:character_images[,false,true]

# Watch logs in another terminal
tail -f log/development.log

4. Batch Large Operations

Break large operations into chunks:

# Instead of all at once
rake granblue:export:character_images

# Process specific batches
rake granblue:export:character_images[3040001000]
rake granblue:export:character_images[3040002000]

5. Check Storage Space

Before bulk downloads:

# Check local space
df -h

# Check S3 usage
aws s3 ls s3://bucket-name/ --recursive --summarize

Scheduling Tasks

Cron Jobs

Example crontab entries:

# Daily wiki data fetch at 2 AM
0 2 * * * cd /app && rake granblue:fetch_wiki_data

# Weekly image download on Sunday at 3 AM
0 3 * * 0 cd /app && rake granblue:download_all_images[character,4]

# Hourly preview generation
0 * * * * cd /app && rake previews:generate_all

Whenever Gem

Using whenever for scheduling:

# config/schedule.rb
every 1.day, at: '2:00 am' do
  rake "granblue:fetch_wiki_data"
end

every :sunday, at: '3:00 am' do
  rake "granblue:download_all_images[character,4]"
end

every 1.hour do
  rake "previews:generate_all"
end

Troubleshooting

Task Not Found

# Check task exists
rake -T | grep task_name

# Ensure in correct directory
pwd  # Should be Rails root

# Load Rails environment
rake -T RAILS_ENV=development

Permission Errors

# Check file permissions
ls -la lib/tasks/

# Fix permissions
chmod +x lib/tasks/*.rake

# Run with bundle
bundle exec rake task_name

Memory Issues

# Increase memory for large operations
export RUBY_HEAP_MIN_SLOTS=500000
export RUBY_GC_MALLOC_LIMIT=90000000

rake granblue:download_all_images[character,2]

Slow Performance

  1. Use parallel processing
  2. Run during off-peak hours
  3. Process in smaller batches
  4. Check network bandwidth
  5. Monitor database connections

Debugging

# Enable debug output
DEBUG=true rake task_name

# Rails console for testing
rails console
> load 'lib/tasks/task_name.rake'
> Rake::Task['task_name'].invoke

# Trace task execution
rake --trace task_name

Creating Custom Tasks

Basic Task Structure

# lib/tasks/custom.rake
namespace :custom do
  desc "Description of the task"
  task :task_name, [:param1, :param2] => :environment do |t, args|
    # Set defaults
    param1 = args[:param1] || 'default'
    param2 = args[:param2] == 'true'

    # Task logic
    puts "Running with #{param1}, #{param2}"

    # Use Rails models
    Model.find_each do |record|
      # Process record
    end
  end
end

Task with Dependencies

namespace :custom do
  task :prepare do
    puts "Preparing..."
  end

  task :cleanup do
    puts "Cleaning up..."
  end

  desc "Main task with dependencies"
  task main: [:prepare, :environment] do
    puts "Running main task"

    # Main logic here

    Rake::Task['custom:cleanup'].invoke
  end
end

Task with Error Handling

namespace :custom do
  desc "Task with error handling"
  task safe_task: :environment do
    begin
      # Risky operation
      process_data
    rescue StandardError => e
      Rails.logger.error "Task failed: #{e.message}"
      Rails.logger.error e.backtrace.join("\n")
      exit 1
    ensure
      # Cleanup
      cleanup_temp_files
    end
  end
end