CS 463G & 663: (Intro to) AI
  • Schedule
  • About
  • Assignments
  • Resources
    • AIMA textbook
    • UK Academic Calendar
    • UK Examination Schedule

Homework 6 Starter Files

Actor-Critic for Buckshot Roulette

Browse every file included in the Homework 6 starter package.

Select a filename below to show its complete contents. These are the same files included in the downloadable ZIP, so this page can be used for a live walkthrough in class.

AssignmentDownload ZIP
  • README.md
  • requirements.txt
  • ai.py
  • buckshot.py
  • main.py
  • table_widget.py
  • train.py
  • check_environment.py
# Homework 6 starter

This package contains the complete Buckshot environment and the interfaces needed for an Actor-Critic agent. The actor, critic, and update rule remain student work.

## Setup

```bash
python -m pip install -r requirements.txt
python check_environment.py
```

## Play the game

Run `python main.py` to open the PySide6 game. In Settings, choose Human or Random for each player. Choose Actor-Critic after completing the agent and running `python train.py` in this folder. Use **Game → New Game** to restart or change players.

Health appears as green segments above each player. The shotgun rests beside the current player, whose name has a gold outline. Items are dealt one at a time into each player's tray. Click an item in the active player's tray to use it, or hover over it to read its effect. Each copy has its own icon and moves from the tray to where it is used. On your turn, click either player's portrait to shoot that player.

The table shows the remaining live and blank shells together in one tray, grouped by type. The shooter reaches for the shotgun and turns it toward the selected player. A live hit flashes the target, shows the damage, and drains health one segment at a time. Each item has its own animation. Cuffs stay on the affected player's wrists until their turn is skipped, and the barrel stays short until the player fires. The move log records each action and its public outcome. If two people share the screen, the other player should look away when a magnifying glass reveals a shell.

The **About** button shows the credits and the educational-use note.

## Files

- `ai.py`: action constants, mask helper, random baseline, and unfinished Actor-Critic classes.
- `main.py`: launches the PySide6 game and player settings.
- `table_widget.py`: draws the illustrated table, interactive portraits, and action effects.
- `buckshot.py`: complete game rules and an 18-value observation interface.
- `train.py`: headless training, evaluation, learning-curve output, and final weights.
- `check_environment.py`: random-agent smoke tests and observation/mask checks.
- `requirements.txt`: required Python packages.

## Train and try your agent

Implement `ActorNetwork`, `CriticNetwork`, and `RLAgent` in `ai.py`. Keep the networks on `self.actor` and `self.critic`, and only update the agent when `training_flag` is true.

Run `python train.py`. It writes `learning_curve.csv`, `learning_curve.png`, and the final network weights in `agent.pt`. Run `python main.py` from the same folder and select Actor-Critic to play with those weights. Use the headless evaluation results for your report.

Upload only `hw6submission.pdf` to Canvas. Include only your `ai.py` and any other source files you wrote or modified in the PDF appendix.
numpy>=1.26
torch>=2.2
matplotlib>=3.8
PySide6>=6.5
import random

import numpy as np
import torch
from torch.distributions import Categorical

HANDSAW = 0
MAGNIFYING_GLASS = 1
CIGARETTE = 2
BEER = 3
HANDCUFFS = 4
SHOOT_SELF = 5
SHOOT_OTHER = 6

ALL_ITEMS = [HANDSAW, MAGNIFYING_GLASS, CIGARETTE, BEER, HANDCUFFS]
ALL_ACTIONS = ALL_ITEMS + [SHOOT_SELF, SHOOT_OTHER]
NUM_OBSERVATIONS = 18
NUM_ACTIONS = 7
MY_ITEMS = slice(13, 18)


def set_seed(seed):
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)


def legal_action_mask(info, device=None):
    mask = torch.ones(NUM_ACTIONS, dtype=torch.bool, device=device)
    item_counts = torch.as_tensor(
        info[MY_ITEMS], dtype=torch.float32, device=device
    )
    mask[:5] = item_counts > 0
    return mask


def masked_policy(logits, info):
    mask = legal_action_mask(info, device=logits.device)
    masked_logits = logits.masked_fill(~mask, float("-inf"))
    return Categorical(logits=masked_logits)


class ActorNetwork(torch.nn.Module):
    def __init__(self, input_dim=NUM_OBSERVATIONS, output_dim=NUM_ACTIONS, hidden_dim=128):
        super().__init__()
        # TODO: define a network that returns output_dim logits.
        self.network = None

    def forward(self, info):
        if self.network is None:
            raise NotImplementedError("Define the actor network")
        return self.network(info)


class CriticNetwork(torch.nn.Module):
    def __init__(self, input_dim=NUM_OBSERVATIONS, hidden_dim=128):
        super().__init__()
        # TODO: define a network that returns one scalar value.
        self.network = None

    def forward(self, info):
        if self.network is None:
            raise NotImplementedError("Define the critic network")
        return self.network(info)


class RandomAgent:
    def choose_action(self, info):
        legal = legal_action_mask(info).nonzero().flatten().tolist()
        return random.choice(legal)

    def see_result(self, result, info, training_flag=False):
        pass


class RLAgent:
    def __init__(self):
        # TODO: create networks, optimizers, hyperparameters, and pending state.
        self.actor = None
        self.critic = None

    def train(self, info, result, terminal=False):
        # TODO: perform one Actor-Critic update from the pending transition.
        raise NotImplementedError("Implement RLAgent.train")

    def choose_action(self, info):
        # TODO: sample from a masked policy and store update information.
        raise NotImplementedError("Implement RLAgent.choose_action")

    def see_result(self, result, info, training_flag=False):
        # A nonzero result marks a terminal win (+1) or loss (-1).
        # TODO: update only when training_flag is True and clear terminal state.
        raise NotImplementedError("Implement RLAgent.see_result")
import random
from dataclasses import dataclass, field

from ai import (
    ALL_ITEMS,
    BEER,
    CIGARETTE,
    HANDCUFFS,
    HANDSAW,
    MAGNIFYING_GLASS,
    SHOOT_OTHER,
    SHOOT_SELF,
)

ITEM_NAMES = {
    HANDSAW: "handsaw",
    MAGNIFYING_GLASS: "magnifying glass",
    CIGARETTE: "cigarette",
    BEER: "beer",
    HANDCUFFS: "handcuffs",
}


@dataclass
class State:
    max_hp: int
    rng: random.Random
    hp: list = field(init=False)
    inventory: list = field(init=False)
    knows_shell: list = field(init=False)
    skip_next: list = field(init=False)
    saw_active: list = field(init=False)
    current_player: int = field(init=False)
    shells: list = field(default_factory=list)
    shell_index: int = 0

    def __post_init__(self):
        self.hp = [self.max_hp, self.max_hp]
        self.inventory = [[0] * 5 for _ in range(2)]
        self.knows_shell = [-1, -1]
        self.skip_next = [False, False]
        self.saw_active = [False, False]
        self.current_player = self.rng.randrange(2)


def remaining_shells(state):
    return state.shells[state.shell_index :]


def reload_shotgun(state):
    total = state.rng.randint(3, 7)
    live = state.rng.randint(1, total - 1)
    state.shells = [1] * live + [0] * (total - live)
    state.rng.shuffle(state.shells)
    state.shell_index = 0
    state.knows_shell = [-1, -1]
    draw_items(state)


def draw_items(state):
    for player in range(2):
        for item in state.rng.sample(ALL_ITEMS, 2):
            if sum(state.inventory[player]) >= 4:
                break
            state.inventory[player][item] += 1


def needs_reload(state):
    remaining = remaining_shells(state)
    return not remaining or 1 not in remaining


def current_shell(state):
    return state.shells[state.shell_index]


def rack_shell(state):
    state.shell_index += 1
    state.knows_shell = [-1, -1]


def observation(state, player):
    other = 1 - player
    remaining = remaining_shells(state)
    info = [
        state.hp[player],
        state.hp[other],
        remaining.count(1),
        remaining.count(0),
        state.knows_shell[player],
        int(state.knows_shell[other] != -1),
        int(state.skip_next[other]),
        int(state.saw_active[player]),
    ]
    info.extend(state.inventory[other])
    info.extend(state.inventory[player])
    assert len(info) == 18
    return info


def legal_actions(state, player):
    actions = [
        item for item in ALL_ITEMS if state.inventory[player][item] > 0
    ]
    return actions + [SHOOT_SELF, SHOOT_OTHER]


def pass_control(state):
    player = state.current_player
    other = 1 - player
    if state.skip_next[other]:
        state.skip_next[other] = False
    else:
        state.current_player = other


def apply_item(state, player, item):
    state.inventory[player][item] -= 1
    other = 1 - player
    if item == HANDSAW:
        state.saw_active[player] = True
    elif item == MAGNIFYING_GLASS:
        state.knows_shell[player] = current_shell(state)
    elif item == CIGARETTE:
        state.hp[player] = min(state.max_hp, state.hp[player] + 1)
    elif item == BEER:
        rack_shell(state)
    elif item == HANDCUFFS:
        state.skip_next[other] = True


def apply_action(state, action):
    """Apply one legal action to a loaded game and return the actor's reward."""
    player = state.current_player
    if action not in legal_actions(state, player):
        raise ValueError(f"P{player} selected illegal action {action}")

    if action in ALL_ITEMS:
        apply_item(state, player, action)
        return 0

    shell = current_shell(state)
    target = player if action == SHOOT_SELF else 1 - player
    if shell == 1:
        damage = 2 if state.saw_active[player] else 1
        state.hp[target] -= damage
    # Every fired shot consumes the handsaw effect, including blanks.
    state.saw_active[player] = False
    rack_shell(state)

    if state.hp[target] <= 0:
        return -1 if target == player else 1
    if shell == 1 or target != player:
        pass_control(state)
    return 0


def describe(state):
    remaining = remaining_shells(state)
    return (
        f"P0 HP={state.hp[0]} items={state.inventory[0]} | "
        f"P1 HP={state.hp[1]} items={state.inventory[1]} | "
        f"shells: {remaining.count(1)} live, {remaining.count(0)} blank | "
        f"turn=P{state.current_player}"
    )


def run_game(agent0, agent1, training=False, seed=None, max_hp=7, verbose=False):
    """Run one game and return winner index, decision count, and final state.

    see_result follows the acting agent's action; the opponent may move before
    that agent's next choose_action. Both agents receive terminal results.
    """
    if seed is not None:
        random.seed(seed)
    state = State(max_hp=max_hp, rng=random.Random(seed))
    agents = [agent0, agent1]
    decisions = 0

    while decisions < 500:
        if needs_reload(state):
            reload_shotgun(state)

        player = state.current_player
        other = 1 - player
        info = observation(state, player)
        action = agents[player].choose_action(info)
        decisions += 1

        if verbose and action in (SHOOT_SELF, SHOOT_OTHER):
            target_name = "self" if action == SHOOT_SELF else f"P{other}"
            shell_name = "live" if current_shell(state) else "blank"
            print(f"P{player} shoots {target_name}: {shell_name}")

        reward = apply_action(state, action)
        if action in ALL_ITEMS:
            agents[player].see_result(0, observation(state, player), training)
            if verbose:
                print(f"P{player} uses {ITEM_NAMES[action]}")
                print(describe(state))
            continue

        if reward:
            winner = other if reward == -1 else player
            agents[player].see_result(
                reward, observation(state, player), training
            )
            agents[other].see_result(
                -reward, observation(state, other), training
            )
            if verbose:
                print(describe(state))
                print(f"P{winner} wins")
            return {"winner": winner, "decisions": decisions, "state": state}

        agents[player].see_result(0, observation(state, player), training)
        if verbose:
            print(describe(state))

    raise RuntimeError("game exceeded 500 decisions")


if __name__ == "__main__":
    from ai import RandomAgent

    result = run_game(RandomAgent(), RandomAgent(), seed=42, verbose=True)
    print("winner:", result["winner"])
import random
import sys
from pathlib import Path

import torch

from PySide6.QtCore import Qt, QTimer, QVariantAnimation
from PySide6.QtGui import QTextOption
from PySide6.QtWidgets import (
    QApplication,
    QComboBox,
    QDialog,
    QFormLayout,
    QFrame,
    QHBoxLayout,
    QLabel,
    QMainWindow,
    QMessageBox,
    QPlainTextEdit,
    QPushButton,
    QVBoxLayout,
    QWidget,
)

from ai import BEER, MAGNIFYING_GLASS, RLAgent, RandomAgent, SHOOT_OTHER, SHOOT_SELF
from buckshot import (
    ITEM_NAMES,
    State,
    apply_action,
    current_shell,
    legal_actions,
    needs_reload,
    observation,
    reload_shotgun,
    remaining_shells,
)
from table_widget import TableWidget, icon, shell_art

ACTION_NAMES = {
    **{item: name.capitalize() for item, name in ITEM_NAMES.items()},
    SHOOT_SELF: "Shoot self",
    SHOOT_OTHER: "Shoot opponent",
}
PLAYER_TYPES = {"Human": None, "Random": RandomAgent, "Actor-Critic": RLAgent}

STYLE = """
QWidget { background: #121917; color: #eae5d5; font-size: 13px; }
QLabel, QToolButton { background: transparent; }
QLabel#eyebrow { color: #b3bfb6; font-size: 11px; font-weight: bold; }
QLabel#turn { color: #f0ce88; font-size: 15px; font-weight: bold; }
QFrame#log { background: #1b2622; border: 1px solid #394b43; border-radius: 9px; }
QPushButton, QToolButton#item {
    background: #26342e; border: 1px solid #506158; border-radius: 7px; padding: 6px 12px;
}
QToolButton#item { padding: 0; border-color: #8b7953; }
QPushButton:hover, QToolButton#item:hover {
    background: #394940; border-color: #e8bd77;
}
QPushButton:focus, QToolButton#item:focus { border-color: #e8bd77; }
QPushButton:disabled, QToolButton#item:disabled {
    background: #1c2723; border-color: #364640; color: #a7b3ae;
}
QPushButton#portrait {
    background: transparent; border: 1px solid transparent; border-radius: 10px; padding: 0;
}
QPushButton#portrait:enabled:hover, QPushButton#portrait:enabled:focus {
    border-color: #e8bd77;
}
QPlainTextEdit {
    background: #1b2622; border: 0; padding: 4px; font-size: 12px;
    selection-background-color: #3c5045; selection-color: #fff4db;
}
"""


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("HW6 Game")
        self.resize(1020, 550)
        self.setStyleSheet(STYLE)
        self.finished = True
        self.after_animation = None

        self.timer = QTimer(self, interval=320, singleShot=True)
        self.timer.timeout.connect(self.play_ai)
        self.animation = QVariantAnimation(self, startValue=0.0, endValue=1.0)
        self.animation.finished.connect(self.animation_done)

        menu = self.menuBar().addMenu("Game")
        self.new_action = menu.addAction("New Game", self.new_game)
        self.new_action.setShortcut("Ctrl+N")

        central = QWidget()
        self.setCentralWidget(central)
        outer = QVBoxLayout(central)
        outer.setContentsMargins(18, 14, 18, 16)
        outer.setSpacing(12)

        layout = QHBoxLayout()
        layout.setSpacing(12)
        outer.addLayout(layout, 1)
        game = QVBoxLayout()
        game.setSpacing(8)
        layout.addLayout(game, 1)
        self.table = TableWidget(self.play)
        self.animation.valueChanged.connect(self.table.set_progress)
        game.addWidget(self.table, 1)

        controls = QHBoxLayout()
        self.turn = self.add_label(controls, "Choose New Game to start", "turn")
        controls.addStretch()
        self.about_button = QPushButton("About")
        self.about_button.clicked.connect(self.show_about)
        controls.addWidget(self.about_button)
        self.new_button = QPushButton("New Game")
        self.new_button.clicked.connect(self.new_game)
        controls.addWidget(self.new_button)
        game.addLayout(controls)

        log = QFrame()
        log.setObjectName("log")
        log.setFixedWidth(226)
        log_layout = QVBoxLayout(log)
        self.add_label(log_layout, "MOVE LOG", "eyebrow")
        self.history = QPlainTextEdit()
        self.history.setAccessibleName("Full move log")
        self.history.setReadOnly(True)
        self.history.setWordWrapMode(QTextOption.WrapMode.WrapAtWordBoundaryOrAnywhere)
        self.history.setHorizontalScrollBarPolicy(Qt.ScrollBarPolicy.ScrollBarAlwaysOff)
        log_layout.addWidget(self.history)
        layout.addWidget(log)

    def add_label(self, layout, text, name):
        label = QLabel(text)
        label.setObjectName(name)
        layout.addWidget(label)
        return label

    def show_about(self):
        resume_ai = self.timer.isActive()
        self.timer.stop()
        about = QMessageBox(self)
        about.setWindowTitle("About HW6 Game")
        about.setTextFormat(Qt.TextFormat.RichText)
        about.setTextInteractionFlags(Qt.TextInteractionFlag.TextBrowserInteraction)
        about.setText(
            "<p>This game is for educational use only.</p>"
            "<p>Course version by Peizhong Ju, with assistance from OpenAI Codex.</p>"
            '<p>Inspired by <a href="https://store.steampowered.com/app/2835570/BuckshotRoulette/">'
            "Buckshot Roulette on Steam</a>, developed by Mike Klubnika "
            "and published by CRITICAL REFLEX.</p>"
        )
        about.exec()
        about.deleteLater()
        if resume_ai:
            self.timer.start()

    def new_game(self):
        self.timer.stop()
        dialog = QDialog(self)
        dialog.setWindowTitle("Settings")
        form = QFormLayout(dialog)
        selectors = [QComboBox(), QComboBox()]
        for player, selector in enumerate(selectors, start=1):
            selector.addItems(PLAYER_TYPES.keys())
            form.addRow(f"Player {player}", selector)
        selectors[1].setCurrentText("Random")
        save = QPushButton("Save Settings")
        save.clicked.connect(dialog.accept)
        form.addRow(save)
        if not dialog.exec():
            if not self.finished:
                self.ready()
            return

        modes = [selector.currentText() for selector in selectors]
        try:
            weights = None
            if "Actor-Critic" in modes:
                checkpoint = Path("agent.pt")
                if not checkpoint.is_file():
                    raise FileNotFoundError("Complete ai.py and run python train.py in this folder first.")
                weights = torch.load(checkpoint, map_location="cpu", weights_only=True)
            agents = [None if mode == "Human" else PLAYER_TYPES[mode]() for mode in modes]
            for mode, agent in zip(modes, agents):
                if mode == "Actor-Critic":
                    agent.actor.load_state_dict(weights["actor"])
                    agent.critic.load_state_dict(weights["critic"])
                    agent.actor.eval()
                    agent.critic.eval()
        except Exception as error:
            QMessageBox.warning(self, "Could not start game", str(error))
            if not self.finished:
                self.ready()
            return

        self.agents = agents
        self.modes = modes
        self.table.reset_items()
        self.state = State(max_hp=7, rng=random.Random())
        self.finished = False
        self.decisions = 0
        self.history.clear()
        self.log(f"Player 1 ({modes[0]}) vs. Player 2 ({modes[1]})")
        self.ready()

    def log(self, message):
        if not self.history.document().isEmpty():
            message = "\n" + message
        self.history.appendPlainText(message)

    def ready(self):
        state = self.state
        player = state.current_player
        if needs_reload(state):
            reload_shotgun(state)
            shells = remaining_shells(state)
            self.log(f"Reload · {shells.count(1)} live, {shells.count(0)} blank · Items dealt")
            self.turn.setText("Reloading…")
            self.present("reload", None, self.ready)
            return

        self.set_navigation_enabled(True)
        human = self.agents[player] is None
        turn = f"Player {player + 1}'s turn"
        self.turn.setText(f"{turn} · Click a portrait to shoot" if human else turn)
        self.set_controls(legal_actions(state, player) if human else ())
        if not human:
            self.timer.start()

    def set_controls(self, legal, player=None):
        if player is None and not self.finished:
            player = self.state.current_player
        self.table.set_legal_actions(player, legal)

    def set_navigation_enabled(self, enabled):
        self.new_action.setEnabled(enabled)
        self.new_button.setEnabled(enabled)
        self.about_button.setEnabled(enabled)

    def present(self, effect, shell, after, *, player=None, target=None, damage=0, item_slot=None):
        self.after_animation = after
        self.set_controls((), player)
        self.set_navigation_enabled(False)
        self.table.update_state(self.state, self.modes, effect, shell, player, target, damage, item_slot)
        if target is not None:
            duration = 1050 + 200 * damage
        elif effect in ITEM_NAMES:
            duration = 1000
        else:
            duration = max(360, 310 * len(self.table.dealt))
        self.animation.setDuration(duration)
        self.table.set_progress(0.0)
        self.animation.start()

    def animation_done(self):
        # Let Qt finish this animation tick before opening a dialog or starting another move.
        QTimer.singleShot(0, self.finish_animation)

    def finish_animation(self):
        after = self.after_animation
        self.after_animation = None
        after()

    def end_game(self, message):
        self.finished = True
        self.timer.stop()
        self.set_controls(())
        self.set_navigation_enabled(True)
        self.turn.setText(message)
        self.log(message)

    def play_ai(self):
        player = self.state.current_player
        try:
            action = self.agents[player].choose_action(observation(self.state, player))
            self.play(action)
        except Exception as error:
            self.end_game(f"Player {player + 1} ({self.modes[player]}): {error}")

    def play(self, action, item_slot=None):
        if self.finished or self.after_animation is not None:
            return
        state = self.state
        player = state.current_player
        other = 1 - player
        agent = self.agents[player]
        hp_before = state.hp.copy()
        cuffs_before = state.skip_next[other]
        target = None
        if action in (SHOOT_SELF, SHOOT_OTHER):
            target = player if action == SHOOT_SELF else other
        shell = current_shell(state) if target is not None or action == BEER else None
        reward = apply_action(state, action)
        self.decisions += 1

        message = f"{self.decisions:03d} · P{player + 1} · {ACTION_NAMES[action]}"
        if shell is not None:
            message += " · Live shell" if shell else " · Blank shell"
        for index, old_hp in enumerate(hp_before):
            change = state.hp[index] - old_hp
            if change:
                message += f" · P{index + 1} {change:+} HP"
        if cuffs_before and not state.skip_next[other]:
            message += f" · P{other + 1} skips turn"
        if agent is not None:
            agent.see_result(reward, observation(state, player), False)
        if reward and self.agents[other] is not None:
            self.agents[other].see_result(-reward, observation(state, other), False)

        def finish_move():
            self.log(message)
            if reward:
                winner = player if reward == 1 else other
                self.end_game(f"Player {winner + 1} wins!")
                self.table.set_legal_actions(winner, ())
            elif self.decisions >= 500:
                self.end_game("Game ended after 500 decisions.")
            elif action == MAGNIFYING_GLASS and agent is None:
                self.show_shell(player, state.knows_shell[player] == 1)
            else:
                self.ready()

        self.turn.setText(f"P{player + 1} · {ACTION_NAMES[action]}")
        damage = hp_before[target] - state.hp[target] if target is not None else 0
        self.present(action, shell, finish_move, player=player, target=target, damage=damage, item_slot=item_slot)

    def show_shell(self, player, live):
        reveal = QMessageBox(self)
        reveal.setAttribute(Qt.WidgetAttribute.WA_DeleteOnClose)
        reveal.setWindowTitle(f"Player {player + 1} · Magnifying glass")
        reveal.setText("Live shell" if live else "Blank shell")
        reveal.setIconPixmap(icon(shell_art, live, 56))
        reveal.setStandardButtons(QMessageBox.StandardButton.Ok)
        reveal.finished.connect(lambda _result: QTimer.singleShot(0, self.ready))
        reveal.open()


def main():
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    window.new_game()
    sys.exit(app.exec())


if __name__ == "__main__":
    main()
"""Small vector illustrations and animations for the Buckshot table."""

import math

from PySide6.QtCore import QPointF, QRectF, QSize, Qt
from PySide6.QtGui import QColor, QFont, QIcon, QPainter, QPen, QPixmap, QPolygonF, QRadialGradient, QTransform
from PySide6.QtWidgets import QPushButton, QToolButton, QWidget

from ai import BEER, CIGARETTE, HANDCUFFS, HANDSAW, MAGNIFYING_GLASS, SHOOT_OTHER, SHOOT_SELF
from buckshot import ITEM_NAMES, remaining_shells

GOLD = "#e8bd77"
LIVE = "#e47b60"
BLANK = "#92b6c4"
MUTED = "#a7b3ae"
HEAL = "#8ace9b"
SHOT_GRAB = 0.26
SHOT_FIRE = 0.50
SHOT_RETURN = 0.28
BEER_EJECT = 0.54
CUFFS_ATTACH = 0.75
CUT_BARREL = 59
JACKETS = ("#355967", "#624638")
GLOVE = "#c5a080"
PLAYER_POSITIONS = ((98, 121), (542, 121))
GUN_PARK_X = (222, 418)
GUN_Y = 116
ITEM_TRAY_X = (19, 485)
ITEM_SLOTS = 4
ITEM_Y = 242

ITEM_HINTS = {
    HANDSAW: "Double the damage of your next live shot.",
    MAGNIFYING_GLASS: "Privately inspect the chambered shell.",
    CIGARETTE: "Recover one health point.",
    BEER: "Eject the chambered shell.",
    HANDCUFFS: "Make the opponent skip their next turn.",
}


def box(painter, x, y, width, height, radius, color, outline=None):
    painter.setBrush(QColor(color))
    painter.setPen(QPen(QColor(outline), 1) if outline else Qt.PenStyle.NoPen)
    painter.drawRoundedRect(QRectF(x, y, width, height), radius, radius)


def line(painter, color, width, x1, y1, x2, y2):
    pen = QPen(QColor(color), width)
    pen.setCapStyle(Qt.PenCapStyle.RoundCap)
    painter.setPen(pen)
    painter.drawLine(QPointF(x1, y1), QPointF(x2, y2))


def polygon(painter, color, points):
    painter.setPen(Qt.PenStyle.NoPen)
    painter.setBrush(QColor(color))
    painter.drawPolygon(QPolygonF([QPointF(x, y) for x, y in points]))


def shell_art(painter, live):
    color = LIVE if live else BLANK
    box(painter, 8, 2, 16, 27, 4, color)
    box(painter, 10, 5, 3, 15, 1, "#efe4d7")
    box(painter, 7, 22, 18, 8, 2, "#c39a5f")
    line(painter, "#f0ce88", 2, 9, 28, 23, 28)


def item_art(painter, item):
    if item == HANDSAW:
        polygon(painter, "#c4d1cb", [(8, 8), (28, 8), (30, 15), (27, 14),
                                   (26, 18), (23, 16), (22, 20), (19, 18),
                                   (18, 22), (15, 20), (13, 24), (8, 23)])
        box(painter, 2, 10, 10, 15, 3, "#b87b49")
        box(painter, 5, 13, 4, 9, 2, "#302a23")
    elif item == MAGNIFYING_GLASS:
        line(painter, "#c08b55", 5, 19, 19, 27, 27)
        box(painter, 3, 3, 19, 19, 9, "#335a63", BLANK)
        line(painter, "#d1e2e7", 2, 7, 12, 11, 8)
    elif item == CIGARETTE:
        box(painter, 2, 16, 28, 7, 2, "#e8ddc4")
        box(painter, 2, 16, 8, 7, 2, "#b58655")
        box(painter, 26, 16, 4, 7, 1, LIVE)
        line(painter, "#9dafa7", 2, 27, 12, 23, 7)
    elif item == BEER:
        box(painter, 7, 4, 18, 24, 3, "#b49158")
        box(painter, 8, 5, 16, 3, 1, "#c9d2c8")
        box(painter, 8, 24, 16, 3, 1, "#c9d2c8")
        polygon(painter, "#e9d8a7", [(17, 10), (12, 17), (16, 17), (15, 22), (20, 15), (16, 15)])
    elif item == HANDCUFFS:
        line(painter, "#aebfbb", 3, 11, 14, 22, 16)
        box(painter, 2, 10, 12, 13, 5, "#536560", "#c4d1cb")
        box(painter, 18, 12, 12, 13, 5, "#536560", "#c4d1cb")
        box(painter, 5, 13, 6, 7, 3, "#202928")
        box(painter, 21, 15, 6, 7, 3, "#202928")


def icon(draw, value, size=32):
    pixmap = QPixmap(size * 2, size * 2)
    pixmap.setDevicePixelRatio(2)
    pixmap.fill(Qt.GlobalColor.transparent)
    painter = QPainter(pixmap)
    painter.setRenderHint(QPainter.RenderHint.Antialiasing)
    painter.scale(size / 32, size / 32)
    draw(painter, value)
    painter.end()
    return pixmap


class HealthMeter(QWidget):
    def __init__(self, parent, player):
        super().__init__(parent)
        self.player = player
        self.hp = self.previous = 0
        self.maximum = 7
        self.progress = 1

    def show_health(self, hp, maximum):
        self.previous, self.hp, self.maximum = self.hp, max(0, hp), maximum
        self.setToolTip(f"{self.hp} / {maximum} health")
        self.setAccessibleName(f"Player {self.player + 1} · {self.toolTip()}")

    def paintEvent(self, event):
        painter = QPainter(self)
        painter.setRenderHint(QPainter.RenderHint.Antialiasing)
        font = painter.font()
        font.setPixelSize(round(self.height() * 0.7))
        font.setBold(True)
        painter.setFont(font)
        painter.setPen(QColor(MUTED))
        label_width = painter.fontMetrics().horizontalAdvance("HP") + 6
        painter.drawText(QRectF(0, 0, label_width, self.height()), Qt.AlignmentFlag.AlignVCenter, "HP")
        width = (self.width() - label_width) / self.maximum
        for point in range(self.maximum):
            color = self.segment_color(point)
            x = label_width + point * width
            box(painter, x + 1, 1, width - 3, self.height() - 2, 3, "#263332", "#485651")
            box(painter, x + 1, 1, width - 3, self.height() - 2, 3, color)

    def segment_color(self, point):
        if self.hp <= point < self.previous:
            lost = self.previous - self.hp
            # Drain from right to left, with one final beat to read the result.
            step = self.progress * (lost + 1) - (self.previous - 1 - point)
            color = QColor(HEAL if step < 0 else LIVE)
            color.setAlphaF(1 - min(1, max(0, step)))
            return color
        if self.previous <= point < self.hp:
            if self.progress < 0.48:
                return QColor("#263332")
            if self.progress < 0.84:
                return QColor("#dcf5bf")
        return QColor(HEAL if point < self.hp else "#263332")


def shotgun_art(painter, muzzle):
    polygon(painter, "#111916", [(-166, 28), (-164, 1), (-125, -11), (-56, 3),
                                 (muzzle - 9, -7), (muzzle + 2, 14), (78, 23), (-32, 22),
                                 (-60, 37), (-100, 41), (-164, 47)])
    polygon(painter, "#8f5e3c", [(-164, -7), (-130, -16), (-86, -9), (-55, -5),
                                 (-55, 14), (-78, 17), (-103, 30), (-164, 36)])
    polygon(painter, "#b88050", [(-163, -5), (-130, -13), (-65, -4), (-65, 3),
                                 (-122, 4), (-160, 17)])
    line(painter, "#4b3428", 4, -164, -5, -164, 34)
    box(painter, -64, -12, 75, 34, 6, "#667a77", "#9aaba3")
    box(painter, -28, -6, 26, 10, 2, "#283938")
    box(painter, -38, 18, 32, 13, 6, "#3e504e", "#748783")
    box(painter, -33, 21, 22, 6, 3, "#1c2926")
    box(painter, 8, -12, muzzle - 11, 13, 3, "#758b89", "#a4b4ab")
    box(painter, 18, 1, muzzle - 26, 10, 3, "#455654")
    box(painter, 18, -1, 77, 19, 4, "#966a46")
    for x in range(26, 91, 8):
        line(painter, "#634932", 2, x, 3, x, 13)
    box(painter, muzzle - 10, -14, 10, 18, 2, "#455654", "#91a29c")


def caption_text(painter, text, color, x, y, size, centered=False):
    font = QFont(painter.font())
    font.setPixelSize(size)
    font.setBold(True)
    painter.setFont(font)
    painter.setPen(QColor(color))
    if centered:
        x -= painter.fontMetrics().horizontalAdvance(text) / 2
    painter.drawText(QPointF(x, y), text)


def draw_player(painter, player, name, selected, damage, progress, reaching, cuffed, inspected):
    color = BLANK if player == 0 else GOLD
    hit = damage > 0 and 0 <= progress < 1
    painter.save()
    if hit:
        away = -1 if player == 0 else 1
        painter.translate(away * 12 * max(0, 1 - progress / 0.40), 0)

    # Two silhouettes with different jackets and visors identify the seats.
    box(painter, -48, 17, 96, 25, 12, "#1a2a25")
    box(painter, -39, -4, 78, 40, 19, JACKETS[player], color)
    box(painter, -22, -42, 44, 49, 17, "#24343b" if player == 0 else "#362c29", color)
    box(painter, -17, -30, 34, 16, 6, "#182523")
    line(painter, color, 3, -11, -23, -3, -23)
    line(painter, color, 3, 3, -23, 11, -23)
    if cuffed:
        box(painter, -26, 15, 20, 13, 5, GLOVE)
        box(painter, 6, 15, 20, 13, 5, GLOVE)
        draw_item(painter, HANDCUFFS, 0, 18, 1.4)
    else:
        if not reaching or player == 0:
            box(painter, -36, 15, 20, 13, 5, GLOVE)
        if not reaching or player == 1:
            box(painter, 16, 15, 20, 13, 5, GLOVE)

    if hit:
        flash = QColor(LIVE)
        flash.setAlphaF(0.64 * max(0, 1 - progress / 0.48))
        box(painter, -39, -4, 78, 40, 19, flash)
        box(painter, -22, -42, 44, 49, 17, flash)
    painter.restore()

    if selected:
        box(painter, -76, 46, 152, 21, 5, "#263a30", GOLD)
    caption_text(painter, name, color, 0, 61, 12, centered=True)
    if inspected:
        draw_item(painter, MAGNIFYING_GLASS, 37, -14, 0.58)
    if cuffed:
        caption_text(painter, "HANDCUFFED", "#e0ddd0", 0, 77, 10, centered=True)
    if hit:
        # Keep the damage readable after the brief impact flash has passed.
        painter.setOpacity(min(1, (1 - progress) * 4))
        y = -83 - 8 * progress
        box(painter, -31, y - 19, 62, 25, 9, "#622f27", LIVE)
        caption_text(painter, f"−{damage} HP", "#ffdfcb", 0, y - 2, 15, centered=True)
        painter.setOpacity(1)


def draw_item(painter, item, x, y, scale=1, angle=0):
    painter.save()
    painter.translate(x, y)
    painter.rotate(angle)
    painter.scale(scale, scale)
    painter.translate(-16, -16)
    item_art(painter, item)
    painter.restore()


class TableWidget(QWidget):
    def __init__(self, on_action):
        super().__init__()
        self.setMinimumSize(560, 237)
        self.shells = (0, 0)
        self.effect = None
        self.shell = None
        self.actor = None
        self.shooter = None
        self.cuffed = self.previous_cuffs = (False, False)
        self.short = self.previous_short = False
        self.target = None
        self.damage = 0
        self.progress = 1
        self.current = None
        self.highlighted = None
        self.modes = (None, None)
        self.inspected = (False, False)
        self.meters = [HealthMeter(self, player) for player in range(2)]
        self.portraits = []
        for player in range(2):
            portrait = QPushButton(self)
            portrait.setObjectName("portrait")
            portrait.setFocusPolicy(Qt.FocusPolicy.StrongFocus)
            portrait.setEnabled(False)
            portrait.clicked.connect(
                lambda checked=False, player=player:
                on_action(SHOOT_SELF if player == self.highlighted else SHOOT_OTHER)
            )
            self.portraits.append(portrait)
        self.reset_items()
        self.item_icons = {}
        for item in ITEM_NAMES:
            art = icon(item_art, item)
            item_icon = QIcon(art)
            item_icon.addPixmap(art, QIcon.Mode.Disabled)
            self.item_icons[item] = item_icon

        self.item_buttons = []
        for player in range(2):
            row = []
            for slot in range(ITEM_SLOTS):
                button = QToolButton(self)
                button.setObjectName("item")
                button.setEnabled(False)
                button.hide()
                button.clicked.connect(
                    lambda checked=False, player=player, slot=slot:
                    on_action(self.inventory[player][slot], slot)
                )
                row.append(button)
            self.item_buttons.append(row)

    def reset_items(self):
        self.inventory = [[None] * ITEM_SLOTS for _ in range(2)]
        self.dealt = []
        self.used_from = None

    def item_position(self, player, slot):
        return ITEM_TRAY_X[player] + 20 + 32 * slot, ITEM_Y

    def resizeEvent(self, event):
        super().resizeEvent(event)
        sx, sy = self.width() / 640, self.height() / 280
        for (x, y), meter, portrait in zip(PLAYER_POSITIONS, self.meters, self.portraits):
            meter.setGeometry(round((x - 66) * sx), round((y - 66) * sy),
                              round(132 * sx), round(14 * sy))
            portrait.setGeometry(round((x - 49) * sx), round((y - 45) * sy),
                                 round(98 * sx), round(87 * sy))
        for player, row in enumerate(self.item_buttons):
            for slot, button in enumerate(row):
                x, y = self.item_position(player, slot)
                button.setGeometry(round((x - 14) * sx), round((y - 14) * sy),
                                   round(28 * sx), round(28 * sy))
                size = round(23 * min(sx, sy))
                button.setIconSize(QSize(size, size))

    def set_legal_actions(self, player, legal):
        self.highlighted = player
        self.update()
        for index, portrait in enumerate(self.portraits):
            action = SHOOT_SELF if index == player else SHOOT_OTHER
            label = f"Shoot Player {index + 1}" + (" (yourself)" if index == player else "")
            portrait.setAccessibleName(label)
            portrait.setToolTip(label)
            portrait.setEnabled(action in legal)
            if portrait.isEnabled():
                portrait.setCursor(Qt.CursorShape.CrossCursor)
            else:
                portrait.unsetCursor()
        for index, row in enumerate(self.item_buttons):
            for slot, button in enumerate(row):
                button.setEnabled(index == player and self.inventory[index][slot] in legal)

    def set_progress(self, progress):
        self.progress = progress
        health_progress = 1 if self.effect == "reload" else progress
        if self.target is not None:
            health_progress = (progress - SHOT_FIRE) / (1 - SHOT_FIRE)
        for meter in self.meters:
            meter.progress = health_progress
            meter.update()
        arrived = int(progress * len(self.dealt))
        in_flight = {(player, slot) for player, slot, item in self.dealt[arrived:]}
        for player, row in enumerate(self.item_buttons):
            for slot, button in enumerate(row):
                visible = self.inventory[player][slot] is not None and (player, slot) not in in_flight
                button.setVisible(visible)
        self.update()

    def update_state(self, state, modes, effect, shell, player, target, damage, item_slot):
        remaining = remaining_shells(state)
        # Display counts only. Grouping by type keeps the actual order hidden.
        self.shells = (remaining.count(1), remaining.count(0))
        self.effect, self.shell = effect, shell
        self.actor = player
        self.current = state.current_player
        self.modes = modes
        self.inspected = tuple(shell != -1 for shell in state.knows_shell)
        for meter, hp in zip(self.meters, state.hp):
            meter.show_health(hp, state.max_hp)
        self.shooter = player if target is not None else None
        self.previous_cuffs, self.cuffed = self.cuffed, tuple(state.skip_next)
        self.previous_short = self.short
        self.short = state.saw_active[state.current_player]
        self.target, self.damage = target, damage
        self.update_items(state, item_slot)

    def update_items(self, state, item_slot):
        self.dealt = []
        self.used_from = None
        if self.effect in ITEM_NAMES:
            slots = self.inventory[self.actor]
            if item_slot is None:
                item_slot = slots.index(self.effect)
            self.used_from = self.item_position(self.actor, item_slot)
            slots[item_slot] = None

        for player, slots in enumerate(self.inventory):
            for item in ITEM_NAMES:
                received = state.inventory[player][item] - slots.count(item)
                for _ in range(received):
                    slot = slots.index(None)
                    slots[slot] = item
                    if self.effect == "reload":
                        self.dealt.append((player, slot, item))

            for item, button in zip(slots, self.item_buttons[player]):
                if item is not None:
                    name = ITEM_NAMES[item].capitalize()
                    button.setIcon(self.item_icons[item])
                    button.setToolTip(f"{name} · {ITEM_HINTS[item]}")
                    button.setAccessibleName(name)

        # Alternate seats while dealing to empty slots at the start.
        self.dealt.sort(key=lambda entry: (entry[1], entry[0]))

    def paintEvent(self, event):
        painter = QPainter(self)
        painter.setRenderHint(QPainter.RenderHint.Antialiasing)
        painter.scale(self.width() / 640, self.height() / 280)
        felt = QRadialGradient(320, 140, 390)
        felt.setColorAt(0, QColor("#345346"))
        felt.setColorAt(1, QColor("#1a2c26"))
        painter.setPen(QPen(QColor("#6d7d5f"), 1))
        painter.setBrush(felt)
        painter.drawRoundedRect(QRectF(1, 1, 638, 278), 13, 13)
        painter.setPen(QPen(QColor("#476354"), 1))
        painter.setBrush(Qt.BrushStyle.NoBrush)
        painter.drawRoundedRect(QRectF(8, 8, 624, 264), 9, 9)

        for player, x in enumerate(ITEM_TRAY_X):
            box(painter, x, 204, 136, 60, 8, "#182923", "#496253")
            color = BLANK if player == 0 else GOLD
            caption_text(painter, f"P{player + 1} ITEMS", color, x + 10, 222, 12)
        self.draw_shell_tray(painter)
        shot_progress = (self.progress - SHOT_FIRE) / (1 - SHOT_FIRE)
        for player, (x, y) in enumerate(PLAYER_POSITIONS):
            painter.save()
            painter.translate(x, y)
            hit = self.damage if player == self.target else 0
            reaching = player == self.shooter and self.progress < 1
            cuffed = self.cuffed[player]
            if self.effect == HANDCUFFS and self.progress < CUFFS_ATTACH:
                cuffed = self.previous_cuffs[player]
            if self.shooter is not None and self.progress < 0.80:
                cuffed = self.previous_cuffs[player]
            name = f"PLAYER {player + 1}"
            if self.modes[player]:
                name += f" · {self.modes[player]}"
            draw_player(painter, player, name, player == self.highlighted, hit,
                        shot_progress, reaching, cuffed, self.inspected[player])
            painter.restore()
        self.draw_shotgun(painter, shot_progress)
        if self.effect in ITEM_NAMES and self.progress < 1:
            self.draw_item_use(painter)
        if self.dealt and self.progress < 1:
            self.draw_deal(painter)

    def draw_shell_tray(self, painter):
        live, blank = self.shells
        firing = self.shooter is not None and self.progress < SHOT_FIRE
        ejecting = self.effect == BEER and self.progress < BEER_EJECT
        if firing or ejecting:
            live += self.shell == 1
            blank += self.shell == 0
        box(painter, 166, 204, 303, 60, 8, "#182923", "#496253")
        caption_text(painter, "SHELLS", MUTED, 178, 222, 12)
        caption_text(painter, f"{live} LIVE", LIVE, 334, 222, 12)
        caption_text(painter, f"{blank} BLANK", BLANK, 392, 222, 12)
        for slot in range(7):
            x = 190 + slot * 37
            box(painter, x, 229, 31, 29, 4, "#253d32")
            if slot < live + blank:
                painter.save()
                painter.translate(x + 3, 231)
                painter.scale(0.78, 0.80)
                if self.effect == "reload":
                    painter.setOpacity(self.progress)
                    painter.translate(0, -14 * (1 - self.progress))
                shell_art(painter, slot < live)
                painter.restore()

    def draw_shotgun(self, painter, shot_progress):
        progress = self.progress
        angle, recoil = -90, 0
        parked_x = GUN_PARK_X[self.current] if self.current is not None else 320
        gun_x = parked_x
        returning = 0
        holding = self.shooter is not None and progress < 1
        if holding:
            # Reach first, then bring the shotgun closer while turning it.
            aim = min(max(0, (progress - SHOT_GRAB) / (SHOT_FIRE - SHOT_GRAB)),
                      1, (1 - progress) / SHOT_RETURN)
            swing = math.sin(aim * math.pi / 2)
            angle += (-90 if self.target == 0 else 90) * swing
            returning = max(0, (progress - (1 - SHOT_RETURN)) / SHOT_RETURN)
            handoff = max(0, (returning - 0.40) / 0.60)
            shooter_x = GUN_PARK_X[self.shooter]
            gun_x = shooter_x + (parked_x - shooter_x) * handoff
            gun_x += (-18 if self.shooter == 0 else 18) * swing
            if shot_progress >= 0:
                recoil = 20 * max(0, 1 - shot_progress / 0.35)

        pose = QTransform()
        pose.translate(gun_x, GUN_Y)
        pose.rotate(angle)
        # A left-facing gun is mirrored so its trigger remains underneath.
        facing_left = self.target == 0 if holding else self.current == 1
        pose.scale(0.52, -0.52 if facing_left else 0.52)
        pose.translate(-recoil, 0)

        if holding:
            seat_x, seat_y = PLAYER_POSITIONS[self.shooter]
            hand_x = seat_x + (26 if self.shooter == 0 else -26)
            hand_y = seat_y + 21
            grip = pose.map(QPointF(-24, 14))
            reach = min(progress / SHOT_GRAB, max(0, 1 - returning / 0.40))
            glove_x = hand_x + (grip.x() - hand_x) * reach
            glove_y = hand_y + (grip.y() - hand_y) * reach
            elbow_x = (hand_x + glove_x) / 2
            elbow_y = (hand_y + glove_y) / 2 + 16 * reach
            sleeve = JACKETS[self.shooter]
            line(painter, sleeve, 17, hand_x, hand_y, elbow_x, elbow_y)
            line(painter, sleeve, 15, elbow_x, elbow_y, glove_x, glove_y)

        painter.save()
        painter.setWorldTransform(pose, True)
        muzzle = self.barrel_end()
        shotgun_art(painter, muzzle)
        if self.target is not None and 0 <= shot_progress < 0.32:
            flash = QColor("#ffd081" if self.shell else BLANK)
            flash.setAlphaF(1 - shot_progress / 0.32)
            painter.translate(muzzle - 173, 0)
            if self.shell:
                polygon(painter, flash, [(173, -10), (210, -28), (206, -11), (256, -4),
                                         (207, 5), (218, 19), (173, 4)])
            else:
                box(painter, 174 + 30 * shot_progress, -14, 25, 21, 10, flash)
        painter.restore()

        if holding:
            box(painter, glove_x - 10, glove_y - 6, 20, 13, 5, GLOVE, "#8c6b55")
            line(painter, "#8c6b55", 1, glove_x + 4, glove_y - 3, glove_x + 4, glove_y + 3)

        shell_progress = shot_progress
        if self.effect == BEER:
            shell_progress = (progress - BEER_EJECT) / (1 - BEER_EJECT)
        if self.shell is not None and 0 <= shell_progress < 1:
            painter.save()
            chamber = pose.map(QPointF(-15, -4))
            painter.translate(chamber.x() - 16 + 46 * shell_progress,
                              chamber.y() - 16 - 34 * math.sin(math.pi * shell_progress))
            painter.rotate(100 * shell_progress)
            painter.setOpacity(1 - shell_progress)
            shell_art(painter, self.shell)
            painter.restore()

    def barrel_end(self):
        if self.effect == HANDSAW and not self.previous_short:
            cut = max(0, min(1, (self.progress - 0.58) / 0.18))
            return 173 - CUT_BARREL * cut
        shortened = self.previous_short if self.shooter is not None and self.progress < 1 else self.short
        return 173 - CUT_BARREL if shortened else 173

    def draw_item_use(self, painter):
        item = self.effect
        progress = self.progress
        seat_x, seat_y = PLAYER_POSITIONS[self.actor]
        inward = 1 if self.actor == 0 else -1
        source_x, source_y = self.used_from
        gun_x = GUN_PARK_X[self.actor]

        if item == HANDCUFFS:
            if progress < CUFFS_ATTACH:
                travel = progress / CUFFS_ATTACH
                other_x, other_y = PLAYER_POSITIONS[1 - self.actor]
                x = source_x + (other_x - source_x) * travel
                y = source_y + (other_y + 18 - source_y) * travel - 38 * math.sin(math.pi * travel)
                draw_item(painter, item, x, y, 0.75 + 0.60 * travel)
            return

        painter.save()
        painter.setOpacity(min(1, (1 - progress) * 7))
        reach = min(1, progress / 0.34)
        scale = 0.75 + 0.60 * reach
        if item == HANDSAW:
            x = source_x + (gun_x - source_x) * reach
            cut_y = GUN_Y - 59
            y = source_y + (cut_y - source_y) * reach
            x += 15 * math.sin(32 * progress) * reach
            draw_item(painter, item, x, y, scale, 12 * reach)
            if 0.48 <= progress <= 0.78:
                for dx, dy in ((-13, -7), (9, -10), (12, 7)):
                    line(painter, GOLD, 2, gun_x + dx / 2, cut_y + dy / 2, gun_x + dx, cut_y + dy)
        elif item == MAGNIFYING_GLASS:
            x = source_x + (gun_x - source_x) * reach
            y = source_y + (GUN_Y - source_y) * reach
            draw_item(painter, item, x + 9 * reach, y + 9 * reach, scale)
            if reach == 1:
                painter.setBrush(Qt.BrushStyle.NoBrush)
                painter.setPen(QPen(QColor(BLANK), 2))
                radius = 15 + 14 * (progress - 0.34)
                painter.drawEllipse(QPointF(x, y), radius, radius)
        else:
            # Bring the cigarette or beer up to the player's visor.
            x = source_x + (seat_x + 13 * inward - source_x) * reach
            y = source_y + (seat_y - 13 - source_y) * reach
            angle = (-58 * inward * reach) if item == BEER else 0
            draw_item(painter, item, x, y, 0.75 + 0.35 * reach, angle)
            if item == CIGARETTE and progress >= 0.38:
                smoke = (progress - 0.38) / 0.62
                painter.setOpacity(1 - smoke)
                for puff in range(3):
                    radius = 3 + 3 * smoke + puff
                    box(painter, x + 10 + 6 * puff * inward, y - 11 - 26 * smoke - 6 * puff,
                        2 * radius, 2 * radius, radius, "#c4d1cb")
        painter.restore()

    def draw_deal(self, painter):
        phase = self.progress * len(self.dealt)
        index = int(phase)
        player, slot, item = self.dealt[index]
        travel = phase - index
        x, y = self.item_position(player, slot)
        box(painter, 282, 20, 76, 30, 7, "#182923", "#496253")
        caption_text(painter, "DEALING", GOLD, 320, 39, 11, centered=True)
        draw_item(painter, item, 320 + (x - 320) * travel,
                  54 + (y - 54) * travel, 0.90 - 0.15 * travel)
import csv

import torch
from matplotlib.figure import Figure

from ai import RLAgent, RandomAgent, set_seed
from buckshot import run_game

TRAINING_GAMES = 1_000
CHECKPOINT_EVERY = 100
EVALUATION_GAMES = 200
BASE_SEED = 42


def evaluate(agent, games=EVALUATION_GAMES, seed_offset=100_000):
    wins = 0
    for game in range(games):
        baseline = RandomAgent()
        seed = seed_offset + game
        if game % 2 == 0:
            result = run_game(agent, baseline, training=False, seed=seed)
            wins += result["winner"] == 0
        else:
            result = run_game(baseline, agent, training=False, seed=seed)
            wins += result["winner"] == 1
    return wins / games


def save_history(history):
    with open("learning_curve.csv", "w", newline="") as output:
        writer = csv.writer(output)
        writer.writerow(["training_games", "winning_rate"])
        writer.writerows(history)

    x_values = [row[0] for row in history]
    y_values = [row[1] for row in history]
    figure = Figure(figsize=(7, 4), layout="tight")
    axes = figure.subplots()
    axes.plot(x_values, y_values, marker="o")
    axes.set(xlabel="Training games", ylabel="Winning rate vs. random agent", ylim=(0, 1))
    axes.grid(alpha=0.25)
    figure.savefig("learning_curve.png", dpi=180)


def main():
    set_seed(BASE_SEED)
    learner = RLAgent()
    history = [(0, evaluate(learner, seed_offset=200_000))]

    for game in range(1, TRAINING_GAMES + 1):
        opponent = RandomAgent()
        if game % 2 == 0:
            run_game(learner, opponent, training=True, seed=BASE_SEED + game)
        else:
            run_game(opponent, learner, training=True, seed=BASE_SEED + game)

        if game % CHECKPOINT_EVERY == 0:
            rate = evaluate(learner, seed_offset=200_000 + game)
            history.append((game, rate))
            print(f"checkpoint {game}: winning rate {rate:.3f}")

    save_history(history)
    torch.save(
        {"actor": learner.actor.state_dict(), "critic": learner.critic.state_dict()},
        "agent.pt",
    )
    print("wrote learning_curve.csv, learning_curve.png, and agent.pt")


if __name__ == "__main__":
    main()
import random
from unittest.mock import patch

from ai import HANDCUFFS, HANDSAW, SHOOT_OTHER, SHOOT_SELF
from ai import RandomAgent, legal_action_mask
from buckshot import State, observation, run_game


class ScriptedAgent:
    def __init__(self, actions):
        self.actions = iter(actions)
        self.observations = []

    def choose_action(self, info):
        self.observations.append(info)
        return next(self.actions)

    def see_result(self, result, info, training_flag=False):
        pass


def scripted_game(shells, hp, actions0, actions1):
    state = State(max_hp=7, rng=random.Random(0))
    state.current_player = 0
    state.hp = list(hp)
    state.shells = shells
    state.inventory[0][HANDSAW] = 1
    state.inventory[0][HANDCUFFS] = 1
    first, second = ScriptedAgent(actions0), ScriptedAgent(actions1)
    with patch("buckshot.State", return_value=state):
        result = run_game(first, second, seed=0)
    return result, first


def main():
    # A live shot is doubled, whether aimed at self or opponent.
    for target, expected_winner in ((SHOOT_SELF, 1), (SHOOT_OTHER, 0)):
        result, _ = scripted_game([1, 0], (2, 2), [HANDSAW, target], [])
        assert result["winner"] == expected_winner
        assert result["decisions"] == 2
        assert not any(result["state"].saw_active)

    # A blank self-shot keeps control but consumes the handsaw bonus.
    result, first = scripted_game(
        [0, 1, 1], (1, 2), [HANDSAW, SHOOT_SELF, SHOOT_OTHER], [SHOOT_OTHER]
    )
    assert first.observations[1][7] == 1  # Bonus exists before the blank.
    assert first.observations[2][7] == 0  # Gone on the retained turn.
    assert result["winner"] == 1  # The following live shot dealt only one.

    # Handcuffs also retain control without preserving the bonus after a blank.
    result, first = scripted_game(
        [0, 1, 1], (1, 2),
        [HANDSAW, HANDCUFFS, SHOOT_OTHER, SHOOT_OTHER], [SHOOT_OTHER]
    )
    assert first.observations[2][7] == 1
    assert first.observations[3][7] == 0
    assert result["winner"] == 1

    for seed in range(10):
        result = run_game(RandomAgent(), RandomAgent(), seed=seed)
        assert result["winner"] in (0, 1)
        state = result["state"]
        for player in (0, 1):
            info = observation(state, player)
            assert len(info) == 18
            mask = legal_action_mask(info)
            assert mask.shape == (7,)
            assert bool(mask[5]) and bool(mask[6])
            assert int(mask.sum()) >= 2
    print("Homework 6 environment passed handsaw checks and 10 random-agent games.")


if __name__ == "__main__":
    main()
← Homework 6 assignmentDownload package →