Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
501942b
Initial commit: Add dataclass_csv2 fork
kraktus Jan 25, 2025
228fb65
Remove stub files, should not longer be necessary as files are typed
kraktus Jan 25, 2025
2ea9ae0
build: Add pyproject.toml with project configuration and dev dependen…
kraktus Jan 25, 2025
faa39cb
refactor: Migrate setup.py configuration to uv-compatible pyproject.toml
kraktus Jan 25, 2025
15575be
refactor: Remove duplicate project section and setup.py
kraktus Jan 25, 2025
4eee06f
wip continue merging pyproject.toml from old setup.py
kraktus Jan 25, 2025
7c9bf71
remove old setup.py and setup.cfg files
kraktus Jan 25, 2025
53ccf40
remove hello.py file that slipped in while initiating with uv
kraktus Jan 25, 2025
cc60adf
re-remove `2` from package name
kraktus Jan 25, 2025
5080b42
Extract `strtobool` function from `distutils.util` module
kraktus Jan 25, 2025
6bbf50a
Add return type to dataclassreader
kraktus Jan 25, 2025
d9f21d8
rename `cls` to `klass` to avoid name collusion with the real `cls` a…
kraktus Jan 25, 2025
83f9509
Support non-list iterable as input to DataclassWriter
kraktus Jan 25, 2025
1bf20b6
strtobool return bool already
kraktus Jan 25, 2025
6a67da7
Fix type error about `key` being unbounded in error message
kraktus Jan 26, 2025
94f604e
fix `accept_whitespaces` decorator typing
kraktus Jan 26, 2025
5a2f55c
Corrected type hint for accept_whitespaces decorator
kraktus Jan 26, 2025
b176b64
finish fixing type issues in test
kraktus Jan 26, 2025
1e084be
try to build the package
kraktus Jan 26, 2025
0865614
fix pyproject.toml to allow building
kraktus Jan 26, 2025
3aa80e2
update readme and compatible python versions
kraktus Jan 26, 2025
fae7639
delete FUNCING.yml
kraktus Jan 26, 2025
706f22c
Add test in CI
kraktus Jan 26, 2025
93dc96b
Update release
kraktus Jan 26, 2025
916c9e0
Update Dependabot configuration for GitHub Actions
kraktus Jan 26, 2025
e731532
Review fixes
dfurtado Nov 19, 2025
f978ac1
Merge branch 'master' into csv2
dfurtado Nov 19, 2025
a6b5c5d
Added new section in the readme
dfurtado Nov 19, 2025
02e60b4
Fix workflow
dfurtado Nov 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Set update schedule for GitHub Actions

version: 2
updates:

- package-ecosystem: "github-actions"
directory: "/"
schedule:
# Check for updates to GitHub Actions every week
interval: "weekly"
36 changes: 36 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Test

on:
push:
branches: ["master"]
paths:
- ".github/workflows/test.yml"
- "**.py"
- "uv.lock"
- "pyproject.toml"
pull_request:
paths:
- ".github/workflows/test.yml"
- "**.py"
- "uv.lock"
- "pyproject.toml"

jobs:
test:
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's great! Exactly what I was going to do. 👍🏽

os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Install uv and set the python version ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install the project
run: uv sync --all-extras --dev
- name: Run tests
run: uv run pytest
42 changes: 21 additions & 21 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
__pycache__
*.pyc
.idea
env/
*.swo
*.swp
*.*~
*.egg
*.egg-info
.#*.*
TAGS
docs
.mypy_cache
.pytest_cache
build
dist
.eggs
.vscode
gmon.out
.vim
pyproject.toml
__pycache__
*.pyc
.idea
env/
*.swo
*.swp
*.*~
*.egg
*.egg-info
.#*.*
TAGS
docs
.mypy_cache
.pytest_cache
build
dist
.eggs
.vscode
gmon.out
.vim
.aider*
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.13
3 changes: 2 additions & 1 deletion AUTHORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@

## Development Lead

* Daniel Furtado <daniel@dfurtado.com>
* Daniel Furtado

## Contributors

* Kraktus
* Nick Schober
* Zoltan Ivanfi
* Alec Benzer
Expand Down
11 changes: 0 additions & 11 deletions MANIFEST.in

This file was deleted.

5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ Dataclass CSV makes working with CSV files easier and much better than working w
using a list of instances of a dataclass.


## Thanks

Thank you to all the amazing contributors who have supported this project over the years, with special thanks to [@kraktus](https://github.com/kraktus) for setting up GitHub Actions, improving automation for package creation, and making numerous code enhancements.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted your work here, I hope it is ok.


## Installation

```shell
Expand Down
7 changes: 0 additions & 7 deletions dataclass_csv/__init__.pyi

This file was deleted.

108 changes: 60 additions & 48 deletions dataclass_csv/dataclass_reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ def strtobool(value: str) -> bool:
return value.lower() in trueValues


T = TypeVar("T")

def _verify_duplicate_header_items(header):
if header is not None and len(header) == 0:
return
Expand All @@ -42,6 +44,17 @@ def _verify_duplicate_header_items(header):
)


def strtobool(value: str) -> bool:
trueValues = ["true", "yes", "t", "y", "on", "1"]

validValues = ["false", "no", "f", "n", "off", "0", *trueValues]

if value.lower() not in validValues:
raise ValueError(f"invalid boolean value {value}")

return value.lower() in trueValues


def is_union_type(t):
if hasattr(t, "__origin__") and t.__origin__ is Union:
return True
Expand All @@ -60,7 +73,7 @@ class DataclassReader(Generic[T]):
def __init__(
self,
f: Any,
cls: Type[T],
klass: Type[T],
fieldnames: Optional[Sequence[str]] = None,
restkey: Optional[str] = None,
restval: Optional[Any] = None,
Expand All @@ -72,10 +85,10 @@ def __init__(
if not f:
raise ValueError("The f argument is required.")

if cls is None or not dataclasses.is_dataclass(cls):
raise ValueError("cls argument needs to be a dataclass.")
if klass is None or not dataclasses.is_dataclass(klass):
raise ValueError("klass argument needs to be a dataclass.")

self._cls = cls
self._cls = klass
self._optional_fields = self._get_optional_fields()
self._field_mapping: Dict[str, Dict[str, Any]] = {}

Expand All @@ -88,7 +101,7 @@ def __init__(
if validate_header:
_verify_duplicate_header_items(self._reader.fieldnames)

self.type_hints = typing.get_type_hints(cls)
self.type_hints = typing.get_type_hints(klass)

def _get_optional_fields(self):
return [
Expand Down Expand Up @@ -120,53 +133,52 @@ def _get_possible_keys(self, fieldname, row):
def _get_value(self, row, field):
is_field_mapped = False

try:
if field.name in self._field_mapping.keys():
is_field_mapped = True
key = self._field_mapping.get(field.name)
else:
key = field.name
if field.name in self._field_mapping.keys():
is_field_mapped = True
key = self._field_mapping.get(field.name)
else:
key = field.name

if key in row.keys():
value = row[key]
else:
if key in row.keys():
value = row[key]
else:
try:
possible_key = self._get_possible_keys(field.name, row)
key = possible_key if possible_key else key
value = row[key]

except KeyError:
if field.name in self._optional_fields:
return self._get_default_value(field)
else:
keyerror_message = f"The value for the column `{field.name}`"
if is_field_mapped:
keyerror_message = f"The value for the mapped column `{key}`"
raise KeyError(f"{keyerror_message} is missing in the CSV file")
else:
if not value and field.name in self._optional_fields:
return self._get_default_value(field)
elif not value and field.name not in self._optional_fields:
raise ValueError(f"The field `{field.name}` is required.")
elif (
value
and field.type is str
and not len(value.strip())
and not self._get_metadata_option(field, "accept_whitespaces")
):
raise ValueError(
(
f"It seems like the value of `{field.name}` contains "
"only white spaces. To allow white spaces to all "
"string fields, use the @accept_whitespaces "
"decorator. "
"To allow white spaces specifically for the field "
f"`{field.name}` change its definition to: "
f"`{field.name}: str = field(metadata="
"{'accept_whitespaces': True})`."
)
except KeyError:
if field.name in self._optional_fields:
return self._get_default_value(field)
else:
keyerror_message = f"The value for the column `{field.name}`"
if is_field_mapped:
keyerror_message = f"The value for the mapped column `{key}`"
raise KeyError(f"{keyerror_message} is missing in the CSV file")

if not value and field.name in self._optional_fields:
return self._get_default_value(field)
elif not value and field.name not in self._optional_fields:
raise ValueError(f"The field `{field.name}` is required.")
elif (
value
and field.type is str
and not len(value.strip())
and not self._get_metadata_option(field, "accept_whitespaces")
):
raise ValueError(
(
f"It seems like the value of `{field.name}` contains "
"only white spaces. To allow white spaces to all "
"string fields, use the @accept_whitespaces "
"decorator. "
"To allow white spaces specifically for the field "
f"`{field.name}` change its definition to: "
f"`{field.name}: str = field(metadata="
"{'accept_whitespaces': True})`."
)
else:
return value
)
else:
return value

def _parse_date_value(self, field, date_value, field_type):
dateformat = self._get_metadata_option(field, "dateformat")
Expand Down Expand Up @@ -231,7 +243,7 @@ def _process_row(self, row) -> T:
transformed_value = (
value
if isinstance(value, bool)
else strtobool(str(value).strip()) == 1
else strtobool(str(value).strip())
)
except ValueError as ex:
raise CsvValueError(ex, line_number=self._reader.line_num) from None
Expand Down
21 changes: 0 additions & 21 deletions dataclass_csv/dataclass_reader.pyi
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great you removed the stubs

This file was deleted.

26 changes: 13 additions & 13 deletions dataclass_csv/dataclass_writer.py
Original file line number Diff line number Diff line change
@@ -1,32 +1,33 @@
import csv
import dataclasses
from typing import Type, Dict, Any, List
from typing import Type, Dict, Any, List, Iterable, Generic, TypeVar
from .header_mapper import HeaderMapper


class DataclassWriter:

T = TypeVar("T")


class DataclassWriter(Generic[T]):
def __init__(
self,
f: Any,
data: List[Any],
cls: Type[object],
data: Iterable[T],
klass: Type[T],
dialect: str = "excel",
**fmtparams: Dict[str, Any],
**fmtparams: Any,
):
if not f:
raise ValueError("The f argument is required")

if not isinstance(data, list):
raise ValueError("Invalid 'data' argument. It must be a list")

if not dataclasses.is_dataclass(cls):
raise ValueError("Invalid 'cls' argument. It must be a dataclass")
if not dataclasses.is_dataclass(klass):
raise ValueError("Invalid 'klass' argument. It must be a dataclass")

self._data = data
self._cls = cls
self._cls = klass
self._field_mapping: Dict[str, str] = dict()

self._fieldnames = [x.name for x in dataclasses.fields(cls)]
self._fieldnames = [x.name for x in dataclasses.fields(klass)]

self._writer = csv.writer(f, dialect=dialect, **fmtparams)

Expand All @@ -48,7 +49,6 @@ def write(self, skip_header: bool = False):
self._fieldnames = self._apply_mapping()

self._writer.writerow(self._fieldnames)

for item in self._data:
if not isinstance(item, self._cls):
raise TypeError(
Expand Down
14 changes: 0 additions & 14 deletions dataclass_csv/dataclass_writer.pyi

This file was deleted.

Loading