-
-
Notifications
You must be signed in to change notification settings - Fork 109
トップページの news セクションを RSS から自動生成 #1704
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+343
−28
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
1d870a4
feat: RSSフィードチェックタスクを追加
nacchan99 d6e64e5
ci: news:fetchタスクを毎朝9:00 JSTに実行するワークフローを追加
nacchan99 3500422
Newsモデルとマイグレーション、テストファイルを追加
nacchan99 6e9c6af
ニュース YAML→DB インポートタスクを追加&news.yml を初期化
nacchan99 55d89c7
Release Phaseに ニュースポイントを追加
nacchan99 d0f3511
fetch_newsタスクのフィードURLを https://coderdojo.jp/#news に修正
nacchan99 e1ab570
feat:fetch_newsタスクのフィードURLを https://news.coderdojo.jp/feed/ に修正
nacchan99 986d4ad
feat: ニュースセクションをDBからの動的表示に切り替え
nacchan99 8ad0b80
refactor: ニュース絵文字の判定ロジックを改善
nacchan99 fd0a4d5
refactor: ニュース表示ロジックを改善
nacchan99 5003621
test: サンプルニュースに絵文字ありのテストケースを追加
nacchan99 c2b1818
fix: 不要な改行を削除
nacchan99 18fa06c
fix: 不要なスペースを削除
nacchan99 57c0d0f
refactor: ニュース取得ロジックをモデルに移動
nacchan99 ca825fd
ワークフローをブランチでも実行
nacchan99 2cf6bcb
fix: ruby-version-file を ruby-version に修正
nacchan99 d058245
chore: update news.yml via GitHub Actions
github-actions[bot] 3a1bece
CoderDojo RSSニュース管理システムの改善
nacchan99 edc41e6
feat: News モデルにバリデーションを追加
nacchan99 21df7e2
refactor: news.yml の構造を整数ID管理に改善
nacchan99 776767c
feat: 絵文字ヘルパーのテストを追加
nacchan99 848eda1
refactor: News ファクトリーを修正
nacchan99 e297078
refactor: ニュースセクションのテストを改善
nacchan99 bdf1337
fix: resolve merge conflicts with main
nacchan99 da8638a
fix: RSS取得処理のセキュリティ改善
nacchan99 81e7ebc
fix: YAMLの安全な読み込みに変更
nacchan99 fc6f171
chore: pull_requestトリガーを削除
nacchan99 7733a59
style: Newsモデルのインデントとバリデーション記述を整理
nacchan99 ac923e8
style: if文のインデント調整と不要なカンマの削除
nacchan99 5f1b8bd
style: safe_openのURL検証を後置unlessで簡潔に記述
nacchan99 cddb4ab
style: コメントとYAML書き出しのインデント整理
nacchan99 0694996
style: ダブルクォートをシングルクォートに変更
nacchan99 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
name: Fetch News | ||
|
||
on: | ||
schedule: | ||
# 毎朝 9:00 JST | ||
- cron: '0 0 * * *' | ||
workflow_dispatch: | ||
|
||
jobs: | ||
fetch: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up Ruby | ||
uses: ruby/setup-ruby@v1 | ||
with: | ||
ruby-version: .ruby-version | ||
bundler-cache: true | ||
|
||
- name: Install dependencies | ||
run: bundle install --jobs 4 --retry 3 | ||
|
||
- name: Run news:fetch task | ||
run: bin/rails news:fetch | ||
|
||
- name: Commit updated news.yml | ||
run: | | ||
git config user.name "github-actions[bot]" | ||
git config user.email "github-actions[bot]@users.noreply.github.com" | ||
git add db/news.yml | ||
if ! git diff --cached --quiet; then | ||
git commit -m "chore: update news.yml via GitHub Actions" | ||
git push | ||
else | ||
echo "No changes in db/news.yml" | ||
fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
class News < ApplicationRecord | ||
scope :recent, -> { order(published_at: :desc) } | ||
|
||
validates :title, presence: true | ||
validates :url, presence: true, | ||
uniqueness: true, | ||
format: { with: /\Ahttps?:\/\/.*\z/i } | ||
validates :published_at, presence: true | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
class CreateNews < ActiveRecord::Migration[8.0] | ||
def change | ||
create_table :news do |t| | ||
t.string :title | ||
t.string :url | ||
t.datetime :published_at | ||
|
||
t.timestamps | ||
end | ||
|
||
add_index :news, :url, unique: true | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
--- | ||
news: | ||
- id: 10 | ||
url: https://news.coderdojo.jp/2025/07/14/233-laptops-to-coderdojo/ | ||
title: 米国系 IT 企業から CoderDojo へ、233 台のノート PC 寄贈 | ||
published_at: Mon, 14 Jul 2025 05:50:31 +0000 | ||
- id: 9 | ||
url: https://news.coderdojo.jp/2025/07/10/dojoletter-vol-86-2025%e5%b9%b405%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.86 2025年05月号 | ||
published_at: Thu, 10 Jul 2025 04:00:07 +0000 | ||
- id: 8 | ||
url: https://news.coderdojo.jp/2025/06/10/dojoletter-vol-85-2025%e5%b9%b404%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.85 2025年04月号 | ||
published_at: Tue, 10 Jun 2025 03:30:18 +0000 | ||
- id: 7 | ||
url: https://news.coderdojo.jp/2025/05/12/dojoletter-vol-84-2025%e5%b9%b403%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.84 2025年03月号 | ||
published_at: Mon, 12 May 2025 04:00:33 +0000 | ||
- id: 6 | ||
url: https://news.coderdojo.jp/2025/04/10/dojoletter-vol-83-2025%e5%b9%b402%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.83 2025年02月号 | ||
published_at: Thu, 10 Apr 2025 03:45:27 +0000 | ||
- id: 5 | ||
url: https://news.coderdojo.jp/2025/04/04/55-laptops-to-coderdojo/ | ||
title: 米国系 IT 企業から CoderDojo へ、55 台のノート PC 寄贈 | ||
published_at: Fri, 04 Apr 2025 10:00:32 +0000 | ||
- id: 4 | ||
url: https://news.coderdojo.jp/2025/03/10/dojoletter-vol-82-2025%e5%b9%b401%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.82 2025年01月号 | ||
published_at: Mon, 10 Mar 2025 04:00:33 +0000 | ||
- id: 3 | ||
url: https://news.coderdojo.jp/2025/02/14/coderdojo-de-nyaicecode/ | ||
title: "\U0001F3B2 ダイス×プログラミング『ニャイス!コード』を、CoderDojo に75台寄贈" | ||
published_at: Fri, 14 Feb 2025 08:24:07 +0000 | ||
- id: 2 | ||
url: https://news.coderdojo.jp/2025/02/10/dojoletter-vol-80-2024%e5%b9%b412%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.80 2024年12月号 | ||
published_at: Mon, 10 Feb 2025 04:00:55 +0000 | ||
- id: 1 | ||
url: https://news.coderdojo.jp/2025/01/14/dojoletter-vol-79-2024%e5%b9%b411%e6%9c%88%e5%8f%b7/ | ||
title: DojoLetter Vol.79 2024年11月号 | ||
published_at: Tue, 14 Jan 2025 03:30:45 +0000 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
require 'rss' | ||
require 'net/http' | ||
require 'uri' | ||
require 'yaml' | ||
require 'time' | ||
require 'active_support/broadcast_logger' | ||
|
||
def safe_open(url) | ||
uri = URI.parse(url) | ||
raise "不正なURLです: #{url}" unless uri.is_a?(URI::HTTP) || uri.is_a?(URI::HTTPS) | ||
|
||
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') do |http| | ||
request = Net::HTTP::Get.new(uri) | ||
response = http.request(request) | ||
response.body | ||
end | ||
end | ||
|
||
namespace :news do | ||
desc 'RSS フィードから最新ニュースを取得し、db/news.yml に書き出す' | ||
task fetch: :environment do | ||
# ロガー設定(ファイル+コンソール出力) | ||
file_logger = ActiveSupport::Logger.new('log/news.log') | ||
console = ActiveSupport::Logger.new(STDOUT) | ||
logger = ActiveSupport::BroadcastLogger.new(file_logger, console) | ||
|
||
logger.info('==== START news:fetch ====') | ||
|
||
# 既存の news.yml を読み込み | ||
yaml_path = Rails.root.join('db', 'news.yml') | ||
existing_news = if File.exist?(yaml_path) | ||
YAML.safe_load(File.read(yaml_path), permitted_classes: [Time], aliases: true)['news'] || [] | ||
else | ||
[] | ||
end | ||
|
||
# テスト/ステージング環境ではサンプルファイル、本番は実サイトのフィード | ||
feed_urls = if Rails.env.test? || Rails.env.staging? | ||
[Rails.root.join('spec', 'fixtures', 'sample_news.rss').to_s] | ||
else | ||
[ | ||
'https://news.coderdojo.jp/feed/' | ||
# 必要に応じて他 Dojo の RSS もここに追加可能 | ||
# 'https://coderdojotokyo.org/feed', | ||
] | ||
end | ||
|
||
# RSS 取得&パース | ||
new_items = feed_urls.flat_map do |url| | ||
logger.info("Fetching RSS → #{url}") | ||
begin | ||
rss = safe_open(url) | ||
feed = RSS::Parser.parse(rss, false) | ||
feed.items.map do |item| | ||
{ | ||
'url' => item.link, | ||
'title' => item.title, | ||
'published_at' => item.pubDate.to_s | ||
} | ||
end | ||
rescue => e | ||
logger.warn("⚠️ Failed to fetch #{url}: #{e.message}") | ||
[] | ||
end | ||
end | ||
|
||
# 既存データをハッシュに変換(URL をキーに) | ||
existing_items_hash = existing_news.index_by { |item| item['url'] } | ||
|
||
# 新しいアイテムと既存アイテムを分離 | ||
truly_new_items = [] | ||
updated_items = [] | ||
|
||
new_items.each do |new_item| | ||
if existing_items_hash.key?(new_item['url']) | ||
# 既存アイテムの更新 | ||
existing_item = existing_items_hash[new_item['url']] | ||
updated_item = existing_item.merge(new_item) # 新しい情報で更新 | ||
updated_items << updated_item | ||
rakuda-san-desu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
else | ||
# 完全に新しいアイテム | ||
truly_new_items << new_item | ||
end | ||
end | ||
|
||
# 既存の最大IDを取得 | ||
max_existing_id = existing_news.map { |item| item['id'].to_i }.max || 0 | ||
|
||
# 新しいアイテムのみに ID を割り当て(古い順) | ||
truly_new_items_sorted = truly_new_items.sort_by { |item| | ||
Time.parse(item['published_at']) | ||
} | ||
|
||
truly_new_items_sorted.each_with_index do |item, index| | ||
item['id'] = max_existing_id + index + 1 | ||
end | ||
|
||
# 更新されなかった既存アイテムを取得 | ||
updated_urls = updated_items.map { |item| item['url'] } | ||
unchanged_items = existing_news.reject { |item| updated_urls.include?(item['url']) } | ||
|
||
# 全アイテムをマージ | ||
all_items = unchanged_items + updated_items + truly_new_items_sorted | ||
|
||
# 日付降順ソート | ||
sorted_items = all_items.sort_by { |item| | ||
Time.parse(item['published_at']) | ||
}.reverse | ||
|
||
File.open('db/news.yml', 'w') do |f| | ||
formatted_items = sorted_items.map do |item| | ||
{ | ||
'id' => item['id'], | ||
'url' => item['url'], | ||
'title' => item['title'], | ||
'published_at' => item['published_at'] | ||
} | ||
end | ||
|
||
f.write({ 'news' => formatted_items }.to_yaml) | ||
end | ||
|
||
logger.info("✅ Wrote #{sorted_items.size} items to db/news.yml (#{truly_new_items_sorted.size} new, #{updated_items.size} updated)") | ||
logger.info('==== END news:fetch ====') | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
require 'yaml' | ||
|
||
namespace :news do | ||
desc 'db/news.yml を読み込んで News テーブルを upsert する' | ||
task import_from_yaml: :environment do | ||
yaml_path = Rails.root.join('db', 'news.yml') | ||
raw = YAML.safe_load(File.read(yaml_path), permitted_classes: [Time], aliases: true) | ||
|
||
# entries を計算 | ||
entries = raw['news'] || [] | ||
|
||
entries.each do |attrs| | ||
news = News.find_or_initialize_by(url: attrs['url']) | ||
news.assign_attributes( | ||
title: attrs['title'], | ||
published_at: attrs['published_at'] | ||
) | ||
news.save! | ||
puts "[news] #{news.published_at.to_date} #{news.title}" | ||
end | ||
rakuda-san-desu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
puts "Imported #{entries.size} items." | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
FactoryBot.define do | ||
factory :news do | ||
sequence(:title) { |n| "Test News Article #{n}" } | ||
sequence(:url) { |n| "https://news.coderdojo.jp/#{n}" } | ||
published_at { 1.day.ago } | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,16 @@ | ||
# -*- coding: utf-8 -*- | ||
require 'rails_helper' | ||
|
||
RSpec.feature "News", type: :feature do | ||
describe "GET /news/2016/12/12/new-backend" do | ||
scenario "Title should be formatted" do | ||
visit "/docs/post-backend-update-history" | ||
expect(page).to have_title "CoderDojo Japan のバックエンド刷新" | ||
RSpec.feature "NewsSection", type: :feature do | ||
let!(:news_item) { create(:news) } | ||
|
||
scenario "ニュースセクションにニュース項目が表示される" do | ||
visit root_path(anchor: 'news') | ||
|
||
within 'section#news' do | ||
expect(page).to have_link(href: news_item.url) | ||
expect(page).to have_content(news_item.title) | ||
expect(page).to have_selector("a[target='_blank']") | ||
end | ||
end | ||
end |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.