Character-Level Superhero Name Generation with TensorFlow

Overview

This project builds a character-level neural text generator that learns naming patterns from a corpus of superhero, supervillain, and other fictional character names, then produces new names one character at a time. The implementation is centered on a Jupyter notebook and follows a TensorFlow workflow that covers data ingestion, tokenizer construction, supervised sequence preparation, model training, and autoregressive text generation.

The implementation focuses on two core technical themes:

• Natural language generation with TensorFlow
• Creating a tokenizer with TensorFlow

At a high level, the notebook trains a next-character prediction model. Given a seed such as d or p, the network repeatedly predicts the most likely next character, appends it to the running sequence, and stops when it emits a dedicated end-of-name token.

This repository is intended to document the notebook-based implementation. The primary artifact is the accompanying superhero_name_generator.ipynb workflow, which contains the full end-to-end experiment.

Project Workflow

The notebook is organized as a step-by-step pipeline:

1. Import the dataset

   • The notebook pulls the source data with:

   git clone https://github.com/am1tyadav/superhero

   • It then loads the raw corpus from superhero/superheroes.txt.

2. Create a character tokenizer

   • A tf.keras.preprocessing.text.Tokenizer is configured with custom filters and split='\n'.
   • The tokenizer is used to build a character vocabulary from the corpus.
   • Two lookup structures are derived:
     • char_to_index: maps each character token to its integer ID
     • index_to_char: reverses the mapping for decoding predictions back into text

3. Convert names into integer sequences

   • name_to_seq(name) converts a single name into a list of integer token IDs.
   • seq_to_name(seq) reconstructs a name from a token sequence while skipping padding tokens.
   • These helpers make it possible to move cleanly between raw text and model-ready numeric input.

4. Generate supervised training examples

   • Each name is expanded into multiple prefix subsequences.
   • For a tokenized name of length n, the notebook creates prefixes of lengths 2 through n.
   • This converts a relatively small name corpus into a much larger next-character prediction dataset.

5. Pad sequences to a fixed length

   • All variable-length prefixes are pre-padded with zeros using tf.keras.preprocessing.sequence.pad_sequences(...).
   • Padding standardizes every sample to the same length so it can be processed efficiently by the neural network.

6. Split features and labels

   • For every padded sequence, the final token becomes the target label y.
   • All preceding tokens become the input sequence x.
   • The dataset is then split into training and validation subsets with train_test_split.

7. Train a neural sequence model

   • The model combines an embedding layer, a causal 1D convolution, max pooling, and an LSTM before a final softmax classifier.
   • Training uses sparse categorical cross-entropy because the target is a single integer class representing the next character.

8. Generate new superhero-style names

   • generate_names(seed) encodes a seed string, pads it to the expected input width, predicts the next character, appends the prediction, and repeats.
   • Decoding stops when the model emits the end-of-name token or when the generation loop reaches its maximum length.

Data and Preprocessing

The dataset contains over 9,000 names collected from superheroes, supervillains, and other fictional characters. The notebook reads the text file as a newline-separated corpus, and example raw entries include:

• jumpa
• doctor fate
• starlight
• isildur
• changeling

In the raw notebook output, names are stored with a trailing tab character (\t). That tab is intentionally preserved and acts as an explicit end-of-name token during training and inference.

Preprocessing Summary

Item	Value	Technical Notes
Dataset file	superhero/superheroes.txt	Pulled from the cloned am1tyadav/superhero repository
Tokenization level	Character-level	Learned with a Keras tokenizer
Stop token	\t	Preserved in the vocabulary and used to terminate generation
Padding token	0	Added by pad_sequences, not part of the learned character vocabulary
Vocabulary size	29	Includes the stop token; excludes padding index 0
Maximum padded length	33	Computed as max_len = 33
Total padded sequences	(88279, 33)	Derived from all generated prefix subsequences
Feature tensor x	(88279, 32)	All tokens except the last one
Target tensor y	(88279,)	Final token in each padded sequence
Training split	(66209, 32)	Produced by train_test_split(x, y)
Validation split	(22070, 32)	Held out for evaluation during training

The character mapping learned in the notebook also shows that the stop token is assigned an ordinary class index inside the vocabulary, while zero remains reserved for padding. This is an important implementation detail because it lets the model learn when a generated name should end instead of relying on a hard-coded character limit alone.

Model Architecture

The notebook defines a compact Sequential model for multiclass next-character prediction. The network receives an input sequence of length 32 (max_len - 1) and predicts one of 29 possible next-character classes.

Layer	Configuration	Output Shape	Parameters
Input	Input(shape=(32,))	(None, 32)	0
Embedding	Embedding(29, 8)	(None, 32, 8)	232
Conv1D	Conv1D(64, 5, strides=1, activation='tanh', padding='causal')	(None, 32, 64)	2,624
MaxPool1D	MaxPool1D(2)	(None, 16, 64)	0
LSTM	LSTM(32)	(None, 32)	12,416
Dense	Dense(29, activation='softmax')	(None, 29)	957

Total trainable parameters: 16,229

This design is small enough to train quickly in a notebook environment while still capturing useful sequential structure:

• The embedding layer projects sparse character IDs into a dense 8-dimensional representation.
• The causal convolution extracts short-range character patterns while preserving left-to-right ordering.
• Max pooling reduces the temporal dimension and compresses local features.
• The LSTM models longer-range dependencies such as common prefixes, suffixes, spacing patterns, and name endings.
• The final softmax layer produces a probability distribution over the next possible character.

Training Configuration and Results

The notebook uses TensorFlow 2.19.0 and compiles the model with settings appropriate for single-step next-token classification.

Training Configuration

Setting	Value
TensorFlow version	2.19.0
Loss	sparse_categorical_crossentropy
Optimizer	adam
Metric	accuracy
Epochs requested	10
Callback	EarlyStopping(monitor='val_accuracy', patience=3)

Final Recorded Results

Metric	Value
Final training accuracy	0.3210
Final validation accuracy	0.3114
Final validation loss	2.2858

The saved notebook output shows that both training and validation accuracy improve steadily across the run, rising from approximately 0.1910 / 0.2301 in the first epoch to 0.3210 / 0.3114 by epoch 10. For a small character-level model trained on names rather than full natural-language sentences, this is a reasonable outcome: the network is learning orthographic structure, common naming fragments, spacing patterns, and end-of-sequence timing, not semantic knowledge about superhero universes.

The notebook also plots training and validation accuracy with Matplotlib, which provides a quick visual check of learning progress across the completed epochs.

Name Generation Logic

Name generation is implemented in the helper function generate_names(seed). The inference loop operates as follows:

1. The seed text is encoded with name_to_seq(seed).
2. The sequence is padded to length 32 using pre-padding and, if necessary, truncated from the left with truncating='pre'.
3. The model predicts a probability distribution over the next character.
4. The next character is selected with greedy decoding via tf.argmax(pred).
5. The predicted character is appended to the seed.
6. The loop stops when the predicted character is \t or when 40 decoding iterations have been executed.

This means the notebook uses deterministic greedy decoding, not temperature-based or random sampling. As a result, repeated runs with the same trained weights and seed are expected to return the same generated output.

Notebook Examples

• Seed d generates dark star
• Seed p generates prack man

These outputs show the model producing names that resemble comic-book naming conventions by recombining familiar character-level patterns learned from the corpus.

Tech Stack

• Python / Jupyter Notebook for the interactive experiment workflow
• TensorFlow / Keras for tokenization, preprocessing, model definition, and training
• scikit-learn for train_test_split
• Matplotlib for visualizing training and validation accuracy
• Git / GitHub for sourcing the external superhero name dataset

Key Takeaways

• This project demonstrates a full character-level text generation pipeline using TensorFlow, from raw text ingestion to neural name synthesis.
• The notebook shows how a tokenizer and lookup dictionaries can bridge human-readable text and model-friendly integer sequences.
• Expanding each name into multiple prefix subsequences transforms a relatively small corpus into a large supervised training set for next-character prediction.
• A lightweight hybrid architecture using embeddings, causal convolution, pooling, and LSTM layers is sufficient to generate plausible superhero-style names from short seeds.
• The implementation is especially useful as a compact educational example of sequence modeling, tokenization, padding, supervised target construction, and autoregressive decoding in TensorFlow.