Final update

.dockerignore

@@ -0,0 +1,4 @@
+# flyctl launch added from .gitignore
+**/.DS_Store
+**/node_modules
+fly.toml

.github/workflows/fly-deploy.yml

@@ -0,0 +1,18 @@
+# See https://fly.io/docs/app-guides/continuous-deployment-with-github-actions/
+
+name: Fly Deploy
+on:
+  push:
+    branches:
+      - main
+jobs:
+  deploy:
+    name: Deploy app
+    runs-on: ubuntu-latest
+    concurrency: deploy-group    # optional: ensure only one action runs at a time
+    steps:
+      - uses: actions/checkout@v4
+      - uses: superfly/flyctl-actions/setup-flyctl@master
+      - run: flyctl deploy --remote-only
+        env:
+          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

.vscode/settings.json

@@ -0,0 +1,5 @@
+{
+    "githubPullRequests.ignoredPullRequestBranches": [
+        "main"
+    ]
+}

README.md

@@ -1,57 +1,94 @@
-# Battlesnake JavaScript Starter Project
+# Reinforcement Learning Battlesnake
 
-An official Battlesnake template written in JavaScript. Get started at [play.battlesnake.com](https://play.battlesnake.com).
+[![Fly Deploy](https://github.com/pieberrykinnie/reinforcement-learning-battlesnake/actions/workflows/fly-deploy.yml/badge.svg)](https://github.com/pieberrykinnie/reinforcement-learning-battlesnake/actions/workflows/fly-deploy.yml)
 
-![Battlesnake Logo](https://media.battlesnake.com/social/StarterSnakeGitHubRepos_JavaScript.png)
+A Battlesnake implementation using Deep Q-Networks (DQN) for reinforcement learning, enhanced with strategic heuristics. Created for the University of Manitoba .devClub Battlesnake Tournament 2023.
 
-This project is a great starting point for anyone wanting to program their first Battlesnake in JavaScript. It can be run locally or easily deployed to a cloud provider of your choosing. See the [Battlesnake API Docs](https://docs.battlesnake.com/api) for more detail. 
+## Overview
 
-[![Run on Replit](https://repl.it/badge/github/BattlesnakeOfficial/starter-snake-javascript)](https://replit.com/@Battlesnake/starter-snake-javascript)
+This project implements a [Battlesnake](https://play.battlesnake.com) using a combination of reinforcement learning and strategic algorithms. The snake learns from its gameplay experiences while using helper algorithms to make informed decisions about food collection, collision avoidance, and board positioning.
 
-## Technologies Used
+### Core Features
 
-This project uses [Node.js](https://nodejs.dev/) and [Express](https://expressjs.com/). It also comes with an optional [Dockerfile](https://docs.docker.com/engine/reference/builder/) to help with deployment.
+- Deep Q-Network implementation using TensorFlow.js
+- Experience replay for stable learning
+- Automated model weight updates
+- Strategic helper algorithms:
+  - Collision prediction
+  - Food prioritization
+  - Edge avoidance
+- Continuous training capability
 
-## Run Your Battlesnake
+## Technical Architecture
 
-Install dependencies using npm
+### Model Structure
 
-```sh
-npm install
-```
+- Input Layer: 125 neurons (11x11 board state + direction vector)
+- Hidden Layers: 2 layers of 64 neurons each with ReLU activation
+- Output Layer: 4 neurons (up, right, down, left) with linear activation
+
+### Key Components
+
+- [`model/calculateReward.js`](model/calculateReward.js): Reward computation (+10 for food, -1 near borders)
+- [`model/convertGameStateToTensor.js`](model/convertGameStateToTensor.js): Game state to tensor conversion
+- [`model/createDQN.js`](model/createDQN.js): Neural network architecture
+- [`model/loadModel.js`](model/loadModel.js): Model loading and initialization
+- [`model/replayBuffer.js`](model/replayBuffer.js): Experience replay implementation
+- [`model/saveModel.js`](model/saveModel.js): Model persistence
+- [`model/trainModel.js`](model/trainModel.js): Q-learning training loop
+
+## Getting Started
+
+### Prerequisites
+
+- Node.js 16 or higher
+- npm package manager
 
-Start your Battlesnake
+### Installation
+
+1. Clone the repository:
 
 ```sh
-npm run start
+git clone https://github.com/pieberrykinnie/reinforcement-learning-battlesnake.git
+cd reinforcement-learning-battlesnake
 ```
 
-You should see the following output once it is running
+1. Install dependencies:
 
 ```sh
-Running Battlesnake at http://0.0.0.0:8000
+npm install
 ```
 
-Open [localhost:8000](http://localhost:8000) in your browser and you should see
+1. Start the server:
 
-```json
-{"apiversion":"1","author":"","color":"#888888","head":"default","tail":"default"}
+```sh
+npm start
 ```
 
-## Play a Game Locally
+The snake will be available at `http://localhost:3000`.
 
-Install the [Battlesnake CLI](https://github.com/BattlesnakeOfficial/rules/tree/main/cli)
-* You can [download compiled binaries here](https://github.com/BattlesnakeOfficial/rules/releases)
-* or [install as a go package](https://github.com/BattlesnakeOfficial/rules/tree/main/cli#installation) (requires Go 1.18 or higher)
+### Running Locally
 
-Command to run a local game
+Use the Battlesnake CLI to test locally:
 
 ```sh
-battlesnake play -W 11 -H 11 --name 'JavaScript Starter Project' --url http://localhost:8000 -g solo --browser
+battlesnake play -W 11 -H 11 --name YourSnakeName --url http://localhost:3000 -g solo -v
 ```
 
-## Next Steps
+## Deployment
+
+This project is configured for deployment on [Fly.io](https://fly.io). The GitHub Actions workflow automatically deploys changes to the main branch.
+
+## Contributors
+
+- Peter Vu - Project Lead & Algorithm Implementation
+- Miah Tayen - TensorFlow Implementation Lead
+
+## Acknowledgments
+
+- University of Manitoba .devClub for organizing the Battlesnake tournament
+- [Battlesnake](https://play.battlesnake.com) for providing the game platform
 
-Continue with the [Battlesnake Quickstart Guide](https://docs.battlesnake.com/quickstart) to customize and improve your Battlesnake's behavior.
+## Project Status
 
-**Note:** To play games on [play.battlesnake.com](https://play.battlesnake.com) you'll need to deploy your Battlesnake to a live web server OR use a port forwarding tool like [ngrok](https://ngrok.com/) to access your server locally.
+The snake achieved second place in the Beginners' Bracket of .devClub's Battlesnake Tournament 2023. Development continues with focus on improving the learning algorithm and CI/CD pipepline.

commit_and_push.sh

@@ -0,0 +1,29 @@
+#!/bin/sh
+
+# Configure Git user information
+git config --global user.email "dmtruong1980@gmail.com"
+git config --global user.name "Quoc Hoang Vu"
+
+# Add, commit, and push the new trained weights
+echo "Adding changes..."
+git add saved_model/
+if [ $? -ne 0 ]; then
+  echo "Failed to add changes"
+  exit 1
+fi
+
+echo "Committing changes..."
+git commit -m "Update trained weights"
+if [ $? -ne 0 ]; then
+  echo "Failed to commit changes"
+  exit 1
+fi
+
+echo "Pushing changes..."
+git push
+if [ $? -ne 0 ]; then
+  echo "Failed to push changes"
+  exit 1
+fi
+
+echo "Changes pushed successfully"