diff --git a/README.md b/README.md index 2fdf292..8518d26 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,6 @@ The xchainClient provides a set of tools to facilitate [cross-chain data bridge](https://github.com/FIL-Builders/onramp-contracts) from any blockchain to Filecoin network. It includes utilities for managing Ethereum accounts, submitting offers, and handling Filecoin deal process. - ## 🚀 Installation Ensure you have **Go** installed. Then, clone the repository and build the project: @@ -14,12 +13,15 @@ go build -o xchainClient ./cmd/xchain.go ``` ### 📌 Configure Environment Variables + 1. Create a `.env` file to store your **XCHAIN_PASSPHRASE** (used for unlocking the keystore to interact with Ethereum compatbile chains): - ```sh - echo "export XCHAIN_PASSPHRASE=your_secure_password" > .env - ``` + + ```sh + echo "export XCHAIN_PASSPHRASE=your_secure_password" > .env + ``` 2. **Source the file** to load the variable into your environment before you run any xchainClient commands: + ```sh source .env ``` @@ -39,6 +41,7 @@ A new command, `generate-account`, allows you to create an Ethereum keystore acc ``` **Example Output** + ``` New Ethereum account created! Address: 0x123456789abcdef... @@ -49,13 +52,15 @@ Keystore File Path: /home/user/onramp-contracts/xchain_key.json 🔹 To enable xChainClient to interact with both the source and destination chains (Filecoin in this case), you need to acquire test tokens for your wallet address on each chain's testnet to cover transaction fees. -- Filecoin calibration faucets: you can find [here](https://docs.filecoin.io/networks/calibration#resources). -- other L1 which you build your application on. +- Filecoin calibration faucets: you can find [here](https://docs.filecoin.io/networks/calibration#resources). +- other L1 which you build your application on. ### 🛠 **config.json** -To run xchainClient for your preference, we will need to update `config.json` with your preferred params. + +To run xchainClient for your preference, we will need to update `config.json` with your preferred params. For example: + - chain config & contracts addresses deployed on that chain. - ClientAddr & PayoutAddr: to pay tx fee and receive payment from Client - OnRampABIPath: copy compiled onramp ABI into this path. @@ -75,6 +80,7 @@ To start the Xchain adapter daemon, run: ``` ## Usages + ### 📡 **offering data with automatic car processing** the `offer-file` command simplifies offering data by automatically: @@ -110,23 +116,40 @@ Example: ### 🔍 **Checking Deal Status** -To check the deal status for a CID: +To check the status of a deal, you can use the following commands: ```sh -./xchainClient client dealStatus -``` +# List all deals +./xchainClient client list-deals -Example: -```sh -./xchainClient client dealStatus bafkreihdwdcef4n 42 +# Check status of a specific deal by UUID +./xchainClient client deal-status ``` +The deal status command will show detailed information about the deal, including: + +- Deal UUID +- Piece CID +- Provider address +- Client address +- Deal size +- Start and end epochs +- Transfer ID +- Retrieval URL +- Current status +- Creation time +- Last status check time + +All deals are stored in JSON format in the `~/.xchain/deals` directory, with each deal saved in a separate file named by its UUID. + ## 🛠️ Configuration ### **Config File (`config.json`)** + The Xchain Client uses a `config.json` file to store its settings. The configuration file should be placed inside the `config/` directory. **Example `config.json`** + ```json { "destination": { @@ -160,37 +183,40 @@ The Xchain Client uses a `config.json` file to store its settings. The configura "TargetAggSize": 67108864, //64MB "MinDealSize": 4194304, //4MB "DealDelayEpochs": 3000, - "DealDuration" : 518400 + "DealDuration": 518400 } ``` ### **Configuration Fields Explained** -| Key | Description | -|------|------------| -| **destination.ChainID** | Ethereum-compatible chain ID for the destination network. | -| **destination.LotusAPI** | Filecoin Lotus API endpoint used for deal tracking. | -| **destination.ProverAddr** | Ethereum address of the prover verifying storage deals. | -| **sources.avalanche.ChainID** | Ethereum-compatible chain ID for the sources network. | -| **sources.avalanche.Api** | WebSocket API for Avalanche network. | -| **sources.avalanche.OnRampAddress** | Avalanche OnRamp contract address. | -| **KeyPath** | Path to the keystore file that contains the Ethereum private key. | -| **ClientAddr** | Ethereum wallet address used for making transactions. | -| **PayoutAddr** | Address where storage rewards should be sent. | -| **OnRampABIPath** | Path to the ABI file for the OnRamp contract. | -| **BufferPath** | Directory where temporary storage is kept before aggregation. | -| **BufferPort** | Port for the buffer service (`5077` by default). | -| **ProviderAddr** | Filecoin storage provider ID. | -| **LighthouseApiKey** | API key for interacting with Lighthouse storage (if applicable). | -| **LighthouseAuth** | Authentication token for Lighthouse. | -| **TransferIP** | IP address for cross-chain data transfer service (`0.0.0.0` for all interfaces). | -| **TransferPort** | Port for the cross-chain data transfer service (`9999` by default). | -| **TargetAggSize** | Specifies the aggregation size for deal bundling, should be power of 2. | -| **MinDealSize** | The minimal aggregation size for a deal, should be power of 2. | -| **DealDelayEpochs** | To calcualte storage deal starting epoch, in blocks. | -| **DealDuration** | To calculate the storage deal validate duration, in blocks. | + +| Key | Description | +| ----------------------------------- | -------------------------------------------------------------------------------- | +| **destination.ChainID** | Ethereum-compatible chain ID for the destination network. | +| **destination.LotusAPI** | Filecoin Lotus API endpoint used for deal tracking. | +| **destination.ProverAddr** | Ethereum address of the prover verifying storage deals. | +| **sources.avalanche.ChainID** | Ethereum-compatible chain ID for the sources network. | +| **sources.avalanche.Api** | WebSocket API for Avalanche network. | +| **sources.avalanche.OnRampAddress** | Avalanche OnRamp contract address. | +| **KeyPath** | Path to the keystore file that contains the Ethereum private key. | +| **ClientAddr** | Ethereum wallet address used for making transactions. | +| **PayoutAddr** | Address where storage rewards should be sent. | +| **OnRampABIPath** | Path to the ABI file for the OnRamp contract. | +| **BufferPath** | Directory where temporary storage is kept before aggregation. | +| **BufferPort** | Port for the buffer service (`5077` by default). | +| **ProviderAddr** | Filecoin storage provider ID. | +| **LighthouseApiKey** | API key for interacting with Lighthouse storage (if applicable). | +| **LighthouseAuth** | Authentication token for Lighthouse. | +| **TransferIP** | IP address for cross-chain data transfer service (`0.0.0.0` for all interfaces). | +| **TransferPort** | Port for the cross-chain data transfer service (`9999` by default). | +| **TargetAggSize** | Specifies the aggregation size for deal bundling, should be power of 2. | +| **MinDealSize** | The minimal aggregation size for a deal, should be power of 2. | +| **DealDelayEpochs** | To calcualte storage deal starting epoch, in blocks. | +| **DealDuration** | To calculate the storage deal validate duration, in blocks. | ### **Multi-Chain Support** + Xchain Client supports interaction with multiple blockchains. Users can configure multiple `sources` to enable cross-chain deal submissions. Supported networks include: + - **Filecoin** - **Avalanche** - **Polygon** @@ -198,11 +224,13 @@ Xchain Client supports interaction with multiple blockchains. Users can configur Each source requires an **API endpoint** and an **OnRamp contract address**, which are specified under the `sources` field in `config.json`. ## 📖 **Additional Notes** + - **Keep your `config.json` file secure** since it contains sensitive information like private key paths and authentication tokens. - **Use strong passwords** when generating Ethereum accounts. - **Regularly back up keystore files** to avoid losing access to funds. ## 💡 Troubleshooting + **Error: "config.json not found"** Ensure the config file is correctly placed in the `config/` directory and named `config.json`. @@ -216,7 +244,9 @@ Ensure the keystore file is correctly generated using `generate-account` and tha Check that your `Api` field in `config.json` is correctly set to a working Ethereum/Web3 provider. ## 🤝 **Contributing** + We welcome contributions! Feel free to submit pull requests or open issues. ## 📜 **License** + This project is licensed under the MIT License. diff --git a/cmd/xchain.go b/cmd/xchain.go index 3a750a2..265e126 100644 --- a/cmd/xchain.go +++ b/cmd/xchain.go @@ -11,6 +11,7 @@ import ( "log" "os" "os/signal" + "time" "golang.org/x/sync/errgroup" @@ -130,6 +131,64 @@ func main() { }, Action: client.OfferCarAction, }, + { + Name: "list-deals", + Usage: "List all recorded deals", + Action: func(cctx *cli.Context) error { + deals, err := deal.ListAllDeals() + if err != nil { + return fmt.Errorf("failed to list deals: %w", err) + } + + if len(deals) == 0 { + fmt.Println("No deals found") + return nil + } + + fmt.Printf("Found %d deals:\n\n", len(deals)) + for _, d := range deals { + fmt.Printf("Deal UUID: %s\n", d.DealUUID) + fmt.Printf(" PieceCID: %s\n", d.PieceCID) + fmt.Printf(" Provider: %s\n", d.Provider) + fmt.Printf(" Size: %d\n", d.Size) + fmt.Printf(" Status: %s\n", d.Status) + fmt.Printf(" Created: %s\n", d.CreatedAt.Format(time.RFC3339)) + fmt.Printf(" Last Checked: %s\n\n", d.LastChecked.Format(time.RFC3339)) + } + return nil + }, + }, + { + Name: "deal-status", + Usage: "Check the status of a specific deal", + ArgsUsage: "", + Action: func(cctx *cli.Context) error { + if cctx.Args().Len() != 1 { + return fmt.Errorf("Usage: deal-status ") + } + + dealUUID := cctx.Args().First() + dealInfo, err := deal.GetDealByUUID(dealUUID) + if err != nil { + return fmt.Errorf("failed to get deal info: %w", err) + } + + fmt.Printf("Deal UUID: %s\n", dealInfo.DealUUID) + fmt.Printf("PieceCID: %s\n", dealInfo.PieceCID) + fmt.Printf("Provider: %s\n", dealInfo.Provider) + fmt.Printf("Client: %s\n", dealInfo.Client) + fmt.Printf("Size: %d\n", dealInfo.Size) + fmt.Printf("Start Epoch: %d\n", dealInfo.StartEpoch) + fmt.Printf("End Epoch: %d\n", dealInfo.EndEpoch) + fmt.Printf("Transfer ID: %d\n", dealInfo.TransferID) + fmt.Printf("Retrieval URL: %s\n", dealInfo.RetrievalURL) + fmt.Printf("Status: %s\n", dealInfo.Status) + fmt.Printf("Created: %s\n", dealInfo.CreatedAt.Format(time.RFC3339)) + fmt.Printf("Last Checked: %s\n", dealInfo.LastChecked.Format(time.RFC3339)) + + return nil + }, + }, }, }, { diff --git a/config/config.json b/config/config.json index cbe50e0..0aafa3c 100644 --- a/config/config.json +++ b/config/config.json @@ -2,13 +2,13 @@ "destination": { "ChainID": 314159, "LotusAPI": "https://api.calibration.node.glif.io", - "ProverAddr": "0x8560C0fAC0EF0547863e1748D15B85a5c3FF4B2f" + "ProverAddr": "0x75c9C9fAC04C696820260CC0bE4201859ff85397" }, "sources": { "avalanche": { "ChainID": 43113, "Api": "wss://api.avax-test.network/ext/bc/C/ws", - "OnRampAddress": "0xb44cc5FB8CfEdE63ce1758CE0CDe0958A7702a16" + "OnRampAddress": "0xeE857540dddB6E6EA10a5c84f57562F11D5Fb47D" } }, "KeyPath": "./config/xchain_key.json", @@ -25,6 +25,5 @@ "TargetAggSize": 67108864, "MinDealSize": 2097152, "DealDelayEpochs": 3000, - "DealDuration" : 518400 - + "DealDuration": 518400 } diff --git a/services/aggregator/aggregator.go b/services/aggregator/aggregator.go index 2ed1fe2..ad65262 100644 --- a/services/aggregator/aggregator.go +++ b/services/aggregator/aggregator.go @@ -130,6 +130,22 @@ type ( LotusTSK = lotustypes.TipSetKey ) +// DealInfo represents the details of a storage deal +type DealInfo struct { + DealUUID uuid.UUID `json:"dealUUID"` + PieceCID cid.Cid `json:"pieceCID"` + Provider address.Address `json:"provider"` + Client address.Address `json:"client"` + Size uint64 `json:"size"` + StartEpoch filabi.ChainEpoch `json:"startEpoch"` + EndEpoch filabi.ChainEpoch `json:"endEpoch"` + TransferID int `json:"transferID"` + RetrievalURL string `json:"retrievalURL"` + Status string `json:"status"` + CreatedAt time.Time `json:"createdAt"` + LastChecked time.Time `json:"lastChecked"` +} + // Function to start the aggregation service func StartAggregationService(ctx context.Context, cfg *config.Config, srcCfg *config.SourceChainConfig) error { aggregator, err := NewAggregator(ctx, cfg, srcCfg) @@ -560,7 +576,28 @@ func (a *aggregator) sendDeal(ctx context.Context, aggCommp cid.Cid, transferID if !resp.Accepted { return fmt.Errorf("deal proposal rejected: %s", resp.Message) } - log.Printf("Deal UUID=%s is sent to miner %s.", dealUuid, a.spActorAddr) + + // Record deal information + dealInfo := DealInfo{ + DealUUID: dealUuid, + PieceCID: aggCommp, + Provider: a.spActorAddr, + Client: filClient, + Size: a.targetDealSize, + StartEpoch: dealStart, + EndEpoch: dealEnd, + TransferID: transferID, + RetrievalURL: url, + Status: "proposed", + CreatedAt: time.Now(), + LastChecked: time.Now(), + } + + if err := a.saveDealInfo(dealInfo); err != nil { + log.Printf("Warning: Failed to save deal info: %v", err) + } + + log.Printf("Deal UUID=%s is sent to miner %s and recorded.", dealUuid, a.spActorAddr) return nil } @@ -827,3 +864,31 @@ func NewLotusDaemonAPIClientV0(ctx context.Context, url string, timeoutSecs int, } var hasV0Suffix = regexp.MustCompile(`\/rpc\/v0\/?\z`) + +// saveDealInfo saves deal information to a JSON file +func (a *aggregator) saveDealInfo(deal DealInfo) error { + // Create .xchain directory if it doesn't exist + homeDir, err := os.UserHomeDir() + if err != nil { + return fmt.Errorf("failed to get home directory: %w", err) + } + + dealsDir := filepath.Join(homeDir, ".xchain", "deals") + if err := os.MkdirAll(dealsDir, 0755); err != nil { + return fmt.Errorf("failed to create deals directory: %w", err) + } + + // Save deal info to a JSON file named by deal UUID + filename := filepath.Join(dealsDir, fmt.Sprintf("%s.json", deal.DealUUID.String())) + dealData, err := json.MarshalIndent(deal, "", " ") + if err != nil { + return fmt.Errorf("failed to marshal deal info: %w", err) + } + + if err := os.WriteFile(filename, dealData, 0644); err != nil { + return fmt.Errorf("failed to write deal info file: %w", err) + } + + log.Printf("Deal info saved to %s", filename) + return nil +} diff --git a/services/deal/storageDeal.go b/services/deal/storageDeal.go index e6c13bd..ed23de7 100644 --- a/services/deal/storageDeal.go +++ b/services/deal/storageDeal.go @@ -2,12 +2,120 @@ package deal import ( "context" + "encoding/json" + "fmt" + "io/ioutil" "log" + "os" + "path/filepath" "time" "github.com/FIL-Builders/xchainClient/config" + "github.com/google/uuid" ) +// DealInfo represents the details of a storage deal +type DealInfo struct { + DealUUID uuid.UUID `json:"dealUUID"` + PieceCID string `json:"pieceCID"` + Provider string `json:"provider"` + Client string `json:"client"` + Size uint64 `json:"size"` + StartEpoch int64 `json:"startEpoch"` + EndEpoch int64 `json:"endEpoch"` + TransferID int `json:"transferID"` + RetrievalURL string `json:"retrievalURL"` + Status string `json:"status"` + CreatedAt time.Time `json:"createdAt"` + LastChecked time.Time `json:"lastChecked"` +} + +// GetDealByUUID retrieves deal information by UUID +func GetDealByUUID(dealUUID string) (*DealInfo, error) { + homeDir, err := os.UserHomeDir() + if err != nil { + return nil, fmt.Errorf("failed to get home directory: %w", err) + } + + filename := filepath.Join(homeDir, ".xchain", "deals", fmt.Sprintf("%s.json", dealUUID)) + data, err := ioutil.ReadFile(filename) + if err != nil { + return nil, fmt.Errorf("failed to read deal file: %w", err) + } + + var dealInfo DealInfo + if err := json.Unmarshal(data, &dealInfo); err != nil { + return nil, fmt.Errorf("failed to unmarshal deal info: %w", err) + } + + return &dealInfo, nil +} + +// ListAllDeals returns a list of all recorded deals +func ListAllDeals() ([]DealInfo, error) { + homeDir, err := os.UserHomeDir() + if err != nil { + return nil, fmt.Errorf("failed to get home directory: %w", err) + } + + dealsDir := filepath.Join(homeDir, ".xchain", "deals") + files, err := ioutil.ReadDir(dealsDir) + if err != nil { + return nil, fmt.Errorf("failed to read deals directory: %w", err) + } + + var deals []DealInfo + for _, file := range files { + if filepath.Ext(file.Name()) != ".json" { + continue + } + + data, err := ioutil.ReadFile(filepath.Join(dealsDir, file.Name())) + if err != nil { + log.Printf("Warning: Failed to read deal file %s: %v", file.Name(), err) + continue + } + + var deal DealInfo + if err := json.Unmarshal(data, &deal); err != nil { + log.Printf("Warning: Failed to unmarshal deal file %s: %v", file.Name(), err) + continue + } + + deals = append(deals, deal) + } + + return deals, nil +} + +// UpdateDealStatus updates the status of a deal +func UpdateDealStatus(dealUUID string, status string) error { + deal, err := GetDealByUUID(dealUUID) + if err != nil { + return err + } + + deal.Status = status + deal.LastChecked = time.Now() + + homeDir, err := os.UserHomeDir() + if err != nil { + return fmt.Errorf("failed to get home directory: %w", err) + } + + filename := filepath.Join(homeDir, ".xchain", "deals", fmt.Sprintf("%s.json", dealUUID)) + dealData, err := json.MarshalIndent(deal, "", " ") + if err != nil { + return fmt.Errorf("failed to marshal deal info: %w", err) + } + + if err := os.WriteFile(filename, dealData, 0644); err != nil { + return fmt.Errorf("failed to write deal info file: %w", err) + } + + return nil +} + // SmartContractDeal continuously runs logic until the context is canceled func SmartContractDeal(ctx context.Context, cfg *config.Config, srcCfg *config.SourceChainConfig) error { log.Println("Starting SmartContractDeal process...")