- 1.Introducation
- 2.Environment Setup
- 3.unittest cases
- 4.Data editing methods
- 5.database importing method
- 6.Manual testing strategy
- 7. Backend Test cases
- 7.1 @test verifying de-duplication
- 7.2 @test verifying replace value
- 7.3 @test verifying change column name
- 7.4 @test verifying split_column_by
- 7.5 @test verifying lower_case_b
- 7.6 @test verifying upper_case_b
- 7.7 @test verifying remove_number_b
- 7.8 @test verifying merge_column_b
- 7.9 @test verifying remove_punctuation_b
- 7.10 @test verifying add_row
- 7.11 @test pop_row
The program under test is a data workbench on sql/csv. The data modification happens in the backend and can be commited into mysql database by the commit function.
- Python 3.9.5
- pip install : dotenv, pylint.
- Vscode with Python 3.9.5 64-bit extension
- windows 10
there is a unittest case in the program file that verifies the functionality of helper function. To do the unit testing, run the unit_test.py in the path backend/src .
There various data editing method which can be separated into two categories -- data query and data cleaning.
- remove_punctuation_b(data, c_num, r_num) # remove punctuation from the data
- merge_column_b(c1, c2, spliter, data, newname) # merge two columns
- remove_number_b(data, c_num, r_num) # remove number from the data
- upper_case_b(data, c_num, r_num) # upper case alphabetic characters.
- lower_case_b(data, c_num, r_num) # lower case alphabetic characters.
- split_column_by_b(data,c1, c2, spliter, cname) # split one column into two
- change_c_name_b(data,cname, new_cname) # change a column's name
- replace_value_b(data, old_val, new_val) # replace a some values with a new value.
- duplicate_eliminate_b(data) # eliminate duplication in the dataset
- add_row(data, dic) # add a new row into the dataset
- pop_row(data, attr, value) # remove certain rows from the dataset
- search_rows(data, attr, value) # finding some rows in the dataset
- get_col(data, col_name) # get a column as a list
- query_assignment2(query_line, mycursor) # query the dataset using query language from assignment2
This program imports data into the mysql database by using the library mysql.connector. mysql.connector can be taken as a medium for communication between the scraping program and the mysql database. After setting the server user and password correctly, we can use the sql cursor from cursor() to manipulate the content of database.
Although unit testing tests some of the functions' functionality, We still need to check if the result of the commit is propagated into the database.
After calling the duplication_elimination(), we can verify the result by using the sql 'count' by checking if some previously duplicated records are reduced to one.
We can check if certain value is replaced by setting the condition to this value, in sql 'where field = the value'. we can verify by checking if the result is empty.
We can verify this by select the new column name.
We can verify this by select the two new column names and the old column, the result of selecting old column name should be empty.
we can verify this by querying data we just lower-cased and check if they are lower-cased.
we can verify this by querying data we just upper-cased and check if they are upper-cased.
we can verify this by querying data we just modified and check if they contain any number.
We can verify this by select the two old column names and the new column, the result of selecting old column names should be empty.
we can verify this by querying data we just modified and check if they contain any punctuations.
we can verify this by querying the row we just added and check if the result matches our input.
we can verify this by querying the row we just poped and check if the result is empty.
the UI of the app should look like this.
user should be able to write text in the code block.
more code block can be added by clicking add code block.
user can compile python code by clicking submit.
the console at the bottom of the code block should be able to print message from standard out. we can test this by printing something.
the console should be able to display the errors and indicates which block causes the error.
User should see a spreadsheet like this after opening the page.
User should be able to edit the spreadsheet directly by clicking on cell.
With correct python code, the compiling result should affect the spreadsheet on the left. for example, this image shows what happen if we add a table to the database.
if there are multiple table in the database, there should be a navigator below the spreadsheet to navigate among sheets.
the column name should appear in the first row.
user can scroll down the spreadsheet.
