How Commodities Trading Firms use General Index & Snowflake in Profit & Loss Reporting

Florence Broderick, Chief Revenue Officer at General Index, was joined by two speakers from Snowflake: Val, an Account Director who works with commodities and trading firms, and Ravi, a Principal Solution Engineer who supports energy and commodities customers. They discussed three data problems trading firms commonly face, then showed how General Index prices can be combined with a firm’s own trade data in Snowflake to calculate profit and loss.

Combining data from many sources

Val said most firms combine data from dozens of sources, including vendors, partners, marketplaces, public datasets, and, increasingly, their own internal systems. Each source has its own schema, update schedule, delivery method, and quality standard. Data teams still spend 70% to 80% of their time cleaning, mapping, deduplicating, and integrating this data. Custom APIs, FTP drops, and inconsistent metadata can turn a single usable dataset into a weeks-long project, which delays analysis and product development.

Licensing and cost control

Data licensing has also become more complex. Flat fees with simple internal-use terms have given way to usage-based pricing, limits on derived data, cloud deployment clauses, privacy rules such as GDPR and CCPA, and new terms covering AI training and resale. Florence said one head of market data at a large energy company had told her that “enterprise pricing” means something different with every vendor.

Surveys suggest market data spending will rise by 1% to 5% this year. Val said a mid-sized trading firm can spend tens of millions of dollars a year on market data, and the largest firms spend hundreds of millions, so even small increases have a direct effect on costs. Many firms also cannot track which teams use which data feeds, which makes those costs harder to manage.

Compliance and consistency

Florence said trading firms are under growing pressure to use compliant data sources. She noted concerns that price assessments based on journalistic judgment can be open to manipulation. General Index follows IOSCO principles and is regulated by the FCA. About 220 contributors send trade data to General Index by API each day, and each index is calculated under a documented methodology that includes a flowchart.

The demo

Ravi showed how a firm can access General Index data through the Snowflake Marketplace, which lists more than 3,000 data products from more than 800 providers. He searched for crude oil prices, selected a General Index product, and added it to his account in a few clicks. Because Snowflake shares data without making a physical copy, the dataset updates automatically each time General Index publishes new prices. There are no pipelines, SFTP transfers, or API calls to maintain.

He then loaded a sample table of trades, with ticker, trade date, buy or sell, traded price, volume, and contract size. Using a short SQL query, he joined the trades to the latest General Index settlement prices and calculated the profit or loss on each position. The query can run on a schedule, for example at the end of each trading day. The results can go straight to risk systems or reporting tools, or arrive by email.

Florence noted that more than 4,200 General Index indexes are available on the Snowflake Marketplace, covering crude oil, refined products, and energy transition markets.

Watch the full webinar below.