Improved column separator detection by ignoring quoted sections

[nicastelo]

Summary:

This pull request enhances the logic used to determine the column separator (delimiter) in CSV files processed by our system. Previously, the method guess_column_separator simply counted occurrences of potential delimiters (such as commas, tabs, semicolons, colons, and pipes) without considering their context. This could lead to misidentification, especially when non-delimiter characters within quoted fields were mistaken for actual delimiters. The updated logic now intelligently ignores delimiters found within quoted sections, leading to more accurate delimiter detection.

Changes:

• Modified the guess_column_separator method to exclude text within quotes when counting delimiter occurrences.
• Added regex-based splitting to remove quoted sections before counting delimiter occurrences in each line.