[deploy] policy 기능 배포

[selnem]

[deploy] policy 기능 배포

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive pipeline for collecting, transforming, and importing government policy news to generate card news slides, adding new API clients, database models, rendering templates, and background schedulers. The review feedback focuses on asynchronous best practices and performance optimizations. Key recommendations include reusing a single httpx.AsyncClient instance for concurrent image downloads, standardizing layout names for consistency, and offloading CPU-bound rendering and synchronous file I/O operations to a thread pool using asyncio.to_thread or to_thread.run_sync to prevent blocking the event loop.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Reusing a single httpx.AsyncClient instance across concurrent image downloads is highly recommended to avoid the overhead of repeatedly establishing TCP/TLS connections. Let's update download_cardnews_image to accept an optional client parameter.

async def download_cardnews_image(
    url: str,
    *,
    timeout: float = 20.0,
    referer: str = "",
    client: httpx.AsyncClient | None = None,
) -> Image.Image | None:
    if not url.startswith(("http://", "https://")):
        return None
    headers = {
        "User-Agent": (
            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
            "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"
        ),
        "Accept": "image/avif,image/webp,image/apng,image/*,*/*;q=0.8",
    }
    if referer:
        headers["Referer"] = referer
    try:
        if client is not None:
            response = await client.get(url, headers=headers)
            response.raise_for_status()
            with Image.open(BytesIO(response.content)) as img:
                return img.convert("RGBA")
        else:
            async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as local_client:
                response = await local_client.get(url, headers=headers)
                response.raise_for_status()
                with Image.open(BytesIO(response.content)) as img:
                    return img.convert("RGBA")
    except (httpx.HTTPError, UnidentifiedImageError, OSError):
        logger.warning("카드뉴스 배경 이미지 다운로드 실패: %s", url)
        return None

[gemini-code-assist]

Reusing a single httpx.AsyncClient instance across concurrent image downloads is highly recommended to avoid the overhead of repeatedly establishing TCP/TLS connections. Let's update download_cardnews_images to reuse a single client.

async def download_cardnews_images(
    urls: list[str],
    *,
    timeout: float = 20.0,
    referer: str = "",
) -> list[Image.Image]:
    import asyncio

    unique_urls: list[str] = []
    seen: set[str] = set()
    for url in urls:
        url = (url or "").strip()
        if not url or url in seen:
            continue
        seen.add(url)
        unique_urls.append(url)

    if not unique_urls:
        return []

    async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
        tasks = [
            download_cardnews_image(url, timeout=timeout, referer=referer, client=client) for url in unique_urls
        ]
        results = await asyncio.gather(*tasks)
    return [img for img in results if img is not None]

[gemini-code-assist]

Using the standardized layout names (template_cover and template_cta) defined in TEMPLATE_LAYOUTS instead of legacy/inconsistent names (cover_big_typo and cta) improves code consistency and prevents potential mismatches in layout checks.

Suggested change

	working = [dict(s) for s in slides[:3]]
	working[0]["layout_type"] = "cover_big_typo"
	if len(working) > 1:
	working[-1]["layout_type"] = "cta"
	working = [dict(s) for s in slides[:3]]
	working[0]["layout_type"] = "template_cover"
	if len(working) > 1:
	working[-1]["layout_type"] = "template_cta"

[gemini-code-assist]

Since get_pipeline_status is called directly by the /status API endpoint, performing synchronous file I/O operations (load_jsonl_rows and load_rows_by_content_id) on the main thread will block the event loop. Offloading these operations to a thread pool using to_thread.run_sync is highly recommended.

Suggested change

	async def get_pipeline_status(self) -> PolicyPipelineStatusResult:
	meta = self._load_sync_meta()
	documents = load_jsonl_rows(POLICY_DOCUMENTS_PATH)
	handoff_by_id = load_rows_by_content_id(POLICY_HANDOFF_PATH)
	async def get_pipeline_status(self) -> PolicyPipelineStatusResult:
	meta = await to_thread.run_sync(self._load_sync_meta)
	documents = await to_thread.run_sync(load_jsonl_rows, POLICY_DOCUMENTS_PATH)
	handoff_by_id = await to_thread.run_sync(load_rows_by_content_id, POLICY_HANDOFF_PATH)

[gemini-code-assist]

Calling prune_pipeline_imported synchronously inside the async sync_pipeline function will block the event loop because it performs synchronous file I/O on POLICY_DOCUMENTS_PATH and POLICY_HANDOFF_PATH. Let's offload this operation to a thread pool using asyncio.to_thread.

Suggested change

	if settings.policy_prune_pipeline_after_import:
	db_ids_before = await ingest_service.get_imported_policy_api_ids()
	prune_pipeline_imported(db_ids_before)
	if settings.policy_prune_pipeline_after_import:
	db_ids_before = await ingest_service.get_imported_policy_api_ids()
	await asyncio.to_thread(prune_pipeline_imported, db_ids_before)

[gemini-code-assist]

Performing synchronous file I/O operations (write_handoff_map) directly on the main thread inside the async transform_documents_jsonl function will block the event loop. Let's offload this operation to a thread pool using asyncio.to_thread.

    await asyncio.to_thread(write_handoff_map, handoff_by_id, dst)

[gemini-code-assist]

Since _maybe_cleanup_local_cardnews is called inside the async generate_cardnews_s3_images function, performing synchronous directory deletion (shutil.rmtree) directly on the main thread will block the event loop. Let's make this function async and offload the deletion to a thread pool using to_thread.run_sync.

Suggested change

	def _maybe_cleanup_local_cardnews(content_id: str) -> None:
	if not settings.policy_cardnews_keep_local_files:
	cleanup_local_cardnews_dir(content_id)
	async def _maybe_cleanup_local_cardnews(content_id: str) -> None:
	if not settings.policy_cardnews_keep_local_files:
	await to_thread.run_sync(cleanup_local_cardnews_dir, content_id)

[gemini-code-assist]

Since _maybe_cleanup_local_cardnews has been updated to be an async function, we should await its execution.

Suggested change

	_maybe_cleanup_local_cardnews(content_id)
	await _maybe_cleanup_local_cardnews(content_id)

[gemini-code-assist]

Performing synchronous file I/O operations (save_slide_image_bytes) directly on the main thread inside the async generate_cardnews_s3_images function will block the event loop. Let's offload this operation to a thread pool using to_thread.run_sync.

                if settings.policy_cardnews_keep_local_files:
                    await to_thread.run_sync(
                        lambda: save_slide_image_bytes(
                            contentid=content_id,
                            slide_no=slide_no,
                            image_bytes=image_bytes,
                            output_dir=target_dir,
                        )
                    )

[gemini-code-assist]

Pillow image rendering (_render_slide_image) and file saving (image.save) are CPU-bound and I/O-bound operations. Executing them directly on the main thread inside the async render_policy_cardnews_slides function will block the event loop. Let's offload these operations to a thread pool using asyncio.to_thread.

Suggested change

	image = _render_slide_image(ctx=ctx)
	image.save(out_path, format="PNG")
	import asyncio
	image = await asyncio.to_thread(_render_slide_image, ctx=ctx)
	await asyncio.to_thread(image.save, out_path, format="PNG")