Fix OSError when displaying Polars DataFrames with old timestamps on Windows #7477
Conversation
…Windows **Fixes #7469** ### Problem When displaying a Polars DataFrame with timestamps before the Unix epoch (1970) or very old dates (e.g., 1902-01-01), marimo would crash on Windows with: ``` OSError: [Errno 22] Invalid argument ``` This occurred in the `_get_bin_values_temporal()` function when calling `datetime.fromtimestamp()` to convert millisecond timestamps back to datetime/date objects for histogram bin visualization. ### Root Cause `datetime.fromtimestamp()` has platform-specific limitations on Windows: - Cannot handle negative timestamps (dates before 1970-01-01) - Cannot handle very old dates that are out of the platform's supported range - This is a known Windows limitation with timestamp conversions ### Solution Modified `_get_bin_values_temporal()` in `narwhals_table.py` to use a more robust approach: - Wrapped `fromtimestamp()` calls in try-except blocks to catch `OSError`, `OverflowError`, and `ValueError` - Added fallback logic using `datetime.timedelta`: `epoch + timedelta(seconds=timestamp)` - This approach works for any date within Python's datetime range (years 1-9999) regardless of platform
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR fixes a Windows-specific crash (OSError) when displaying Polars DataFrames containing timestamps before the Unix epoch (1970-01-01) or very old dates. The fix wraps datetime.fromtimestamp() calls in try-except blocks with a fallback using datetime.timedelta arithmetic, which works across all platforms and date ranges supported by Python's datetime.
Key changes:
- Added try-except error handling for
OSError,OverflowError, andValueErrorwhen callingfromtimestamp() - Implemented fallback logic using
epoch + timedelta(seconds=timestamp)for dates/datetimes that fail withfromtimestamp() - Added comprehensive test coverage for old dates (pre-1970) that previously caused crashes on Windows
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| marimo/_plugins/ui/_impl/tables/narwhals_table.py | Added try-except blocks with timedelta-based fallback for both Date and Datetime bin value calculations in _get_bin_values_temporal() |
| tests/_plugins/ui/_impl/tables/test_narwhals.py | Added new test class TestOldDates with a test case that reproduces the issue with 1902-01-01 timestamps |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| epoch = datetime.datetime( | ||
| 1970, 1, 1, tzinfo=datetime.timezone.utc | ||
| ) | ||
| bin_end = epoch + datetime.timedelta( | ||
| seconds=bin_end / ms_time | ||
| ) | ||
| # Remove timezone to match fromtimestamp behavior | ||
| bin_end = bin_end.replace(tzinfo=None) |
There was a problem hiding this comment.
The fallback logic creates a UTC-based datetime while datetime.fromtimestamp() creates a local timezone datetime. This inconsistency can cause the bin_end values to differ depending on which code path is taken.
To match the behavior of fromtimestamp(), the epoch should be defined without timezone (using local time), or use datetime.utcfromtimestamp() for consistent UTC behavior. However, note that utcfromtimestamp() is deprecated in Python 3.12+.
The recommended approach is to use datetime.fromtimestamp(timestamp, tz=timezone.utc).replace(tzinfo=None) for both the success and fallback paths to ensure consistent UTC interpretation.
| epoch = datetime.datetime( | |
| 1970, 1, 1, tzinfo=datetime.timezone.utc | |
| ) | |
| bin_end = epoch + datetime.timedelta( | |
| seconds=bin_end / ms_time | |
| ) | |
| # Remove timezone to match fromtimestamp behavior | |
| bin_end = bin_end.replace(tzinfo=None) | |
| bin_end = datetime.datetime.fromtimestamp( | |
| bin_end / ms_time, tz=datetime.timezone.utc | |
| ).replace(tzinfo=None) |
| epoch = datetime.datetime( | ||
| 1970, 1, 1, tzinfo=datetime.timezone.utc | ||
| ) | ||
| bin_end_dt = epoch + datetime.timedelta( | ||
| seconds=bin_end / ms_time | ||
| ) | ||
| bin_end = bin_end_dt.date() |
There was a problem hiding this comment.
The fallback logic creates a UTC-based date while datetime.date.fromtimestamp() creates a local timezone date. This inconsistency can cause the bin_end values to differ depending on which code path is taken.
To match the behavior of fromtimestamp(), the epoch should be defined without timezone (using local time). The recommended approach is to use datetime.fromtimestamp(timestamp, tz=timezone.utc).date() for both the success and fallback paths to ensure consistent UTC interpretation, or use a naive epoch like datetime.datetime(1970, 1, 1) for local time interpretation.
Fixes #7469
Problem
When displaying a Polars DataFrame with timestamps before the Unix epoch (1970) or very old dates (e.g., 1902-01-01), marimo would crash on Windows with:
This occurred in the
_get_bin_values_temporal()function when callingdatetime.fromtimestamp()to convert millisecond timestamps back to datetime/date objects for histogram bin visualization.Root Cause
datetime.fromtimestamp()has platform-specific limitations on Windows:Solution
Modified
_get_bin_values_temporal()innarwhals_table.pyto use a more robust approach:fromtimestamp()calls in try-except blocks to catchOSError,OverflowError, andValueErrordatetime.timedelta:epoch + timedelta(seconds=timestamp)