Experiments in Market Data Visualization

Market data visualization demo.

---

Visuals

Order book 3D surface plot.

Valley-shaped surface showing cumulative bids and asks.

X-Axis: Time

Y-Axis: Price

Z-Axis: Amount

Color: Order type (green = bid, red = ask). Shade gets darker the further price is away from market price.

Graph visualizes order book depth and bid-ask spread. Animation shows changes in these parameters over time.

At the X-end of the plot area is a 2-D plot showing the market depth animated in place. It is a simplified representation of the last slice of the order book surface plot.

Mid price line.

2-D line plot that hovers above the order book surface plot and traces the mid price.

X-Axis: Time

Y-Axis: Price

Trades.

Trades rise from the order book surface chart as animated colored cylinders.

Long trades are colored green and short trades are colored red.

X-Axis: Trade execution time

Y-Axis: Trade price

Dimension (height of cylinder): Trade amount

Color: Trade type (green = long, red = short)

Once trades have risen up to the level of the mid price line, they start moving to a grid at the Y-end of the plot area forming a traditional 2-D trade volume plot along the X-axis (time axis).

---

Audio

The sound that plays during the animation represents trading activity. Every block of 50 trades is transformed into a single note.

Pitch: Weighted average trade price

Velocity (amplitude): Average trade amount (log-transformed)

Duration: Time between execution time of first and last trade in block

Increased trading activity results in more notes being played per second. This is very audible around the 22 second mark of the video when trading activity and price increase significantly.

The Intrinsic Value of Crypto Network Tokens

Even informed commentators often argue that crypto network tokens (bitcoin, Ethereum ether, etc.) have no "real" value or utility. The "coins" only have value because a group of people accept them as a medium of exchange or, more importantly, they have the speculative value of a du-jour Ponzi asset.

As I have argued in a previous post, a crypto network is technical infrastructure; a network of compute and storage devices. Crypto network token confer the right to consume compute and storage resources on that network.

Consider following questions:

What would you be willing to pay to store 1 GB of data for an indefinite period of time on a very secure decentralized and distributed compute network that is always available and can be accessed from any internet-connected device? If the answer is "more than 0 dollars" then crypto network tokens such as ether have positive intrinsic value.

What does it cost to store that amount of data and run applications for a "very long" period on a platform such as Microsoft Azure or Amazon AWS each instance replicated "many" times and how much do you value decentralization and quantifiable security?

The intrinsic value of the network token is correlated to the security of the network, e.g. hash rate, and the level of distribution, e.g. number of nodes. If there are multiple comparable networks, for example multiple alternative Ethereum networks, then the token for the network with the highest level of difficulty and the largest number of nodes should command the highest price if the networks are otherwise comparable, e.g. in terms of transaction throughput and confirmation speed.

The fact that an unlimited number of Ethereum networks can be created does not diminish the value of the existing networks and their respective tokens unless resources are reallocated.

There should be a (set of) equilibrium price(s) where users' willingness to pay for compute and storage at the given level of security and distribution equals the cost of maintaining the network (aka "mining" costs") plus some margin. The relationship between price, mining hash power, and network difficulty should ensure that the token price closely reflects mining cost, i.e. network tokens should have an administrative price.

However, if networks tokens are used as a means of payment to settle transaction or as a store of value and speculative investment then the token price could be decoupled from its intrinsic value.

I will explore this question more deeply in a future post. For now let's have a look at the relationship between intrinsic value of ether and the strength of the Ethereum network.

For simplicity, we will express intrinsic value of the Ethereum network token as the number of bytes that can be stored on the network for one ether and we will use hash power as a proxy for network strength.

The Fall of Intrinsic Value and the Rise of Extrinsic Value

There is a clear inverse relationship between the data storage value of one ether and network hash power. The correlation over the observed time frame was -0.7.

The amount of storage one ether buys on the network has decreased because the gas price has increased. One possible interpretation is that miners need to be compensated for providing more hash power with a higher gas price all other things being equal.

However, miners compensation also depends on the token price at which ether can be converted to fiat currency, assuming that miners pay their expenses in fiat currency. Interestingly, the network token price has also increased significantly over the observed period.

The chart shows that in relative terms, ether storage value has fallen at a higher rate than the network hash rate has increased while the network token price has inflated substantially. This means that the intrinsic value of ether as defined in this post has fallen while extrinsic value has risen.

Protecting APIs and Users with OAuth and Asymmetric Authentication

Introduction

OAuth2 dropped per-call cryptographic singing of requests in favor of easier to use, but arguably less secure access tokens. Users whose access tokens are intercepted can be impersonated as long as the token is valid. This would be much harder if every request had to be signed using asymmetric cryptography allowing resource servers to authenticate each API call before granting authorization.

While this may be cumbersome for some types of applications, it is trivial for applications that utilize blockchain networks as the ability to create and verify digital signatures using public key cryptography is innate to those applications.

This article demonstrated how adding asymmetric authentication to your OAuth2 flow improves both user experience and security.

Authentication/Authorization Flows

First, let us review a conventional OAuth2/OpenID Connect flow without asymmetric authentication:

OAuth2 with OpenID Connect

1 User identity is unknown when the web application is first accessed. The web application redirects to the web front-end of the identity provider via a sign-in button.

2 The user leaves the web application and is presented with the login screen of the identity provider.

3 The user id and password along with user claim and scope grants are sent to the identity provider’s backend server and validated. If validation succeeds, the user is redirected back to the web application along with either a token that will allow the web application to request an id and access token from the identity provider or it receives those tokens directly with the redirect.

4 Back in the web application, the user identity is now known via information encoded in the id token. Authorization for API calls is obtained by including the access token with every request to the web API. The web API will have to access the identity server at least once in order to validate the access token presented by the web client, but it will not have to do that for every request. Authorization happens in the web API, i.e. the web API decides what resources are available based on information contained in the access token. In addition to user id, the access token can contain user claims such as age, country of residence etc.

Key

The advantage of OAuth2 with OpenID Connect is that user credentials do not flow through the web application but are only known to the identity provider which is completely separate from the web application and associated API. Additionally, any number of web applications can utilize the same identity provider for authentication and users can manager access grants for all those applications centrally in one place.

However, the only time the user is actually authenticated is when the user id and password is validated by the identity server. All subsequent requests to the web API are not authenticated and authorization is granted purely based on the presence of the JWT access token. If the access token is leaked or intercepted, then the user can be impersonated simply by including the token with forged requests.

Adding asymmetric authentication to the flow eliminates these shortcomings:

Asymmetric Authentication with Authorization Token

1 With asymmetric authentication, there is no need to redirect to a login screen because users are authenticated from the very first interaction with the web application.

2 Id and access tokens are requested automatically without user interaction from the identity server using a cryptographically signed message that proves the user’s identity.

3 User id and access token are obtained by the web application and used for claims-based authorization only (user claims are encoded in the tokens). When making an API request, a cryptographic signature is included along with the access token. The signature is unique for each request and therefore the web API is able to authenticate every request. Once authenticated, access to API resources is authorized based on the provided access token.

Key

There are several advantages of this flow over the previous OAuth2/OpenId Connect flow.

The user does not have to leave the web application in order to sign in - in fact there is no need to log in at all. When opening the web application, the user is authenticated automatically, and the full application and user data become available immediately.

Authorization works as before, but it is no longer possible to use leaked or intercepted access tokens for impersonation. Every call to the web API is authenticated. The access token alone is insufficient to gain access to API resources and should be more appropriately called "authorization token".

Signature validation is a compute-bound operation that does not create additional network traffic.

In short, adding Asymmetric Authentication to the OAuth flow improves both user experience and security without reducing scalability.

The flow can be further simplified if the id and authorization token (aka access token) are not needed for authorization, i.e. if authorization does not rely on user claims encoded in the tokens but is purely determined by the user's identity:

Asymmetric Authentication without Authorization Token

1 If the web API does not depend on user claims included in the JWT access token then there is no need to obtain one from the identity server. The cryptographically signed request must include the public key in order for the web API to validate the signature. Therefore, the authenticated user's identity (public key) is provided to the web API with every request. In many scenarios this will be enough for managing authorization, for example if the user profile has to be loaded anyway in order to determine the user's registration status.

This flow is also very well suited for direct API access from command line clients:

Sample Implementation for .NET Web API and VueJS Web Client

In your ASP.NET Core Web API, import the ProDerivatives.AsymmetricAuthentication NuGet package. As we will be using Ethereum signatures, also import the ProDerivatives.Ethereum NuGet package.

Add below code to you Startup class:

Enable asymmetric authentication in ConfigureService method:

services.AddAuthentication(AsymmetricAuthenticationDefaults.AuthenticationScheme)
    .AddAsymmetricAuthentication(options =>
    {
        options.SignatureTokenRetriever = TokenRetrieval.FromAuthenticationHeader();
        options.SignatureValidator = VerifySignature();
    });

Verify signature, in this case Ethereum signatures using ProDerivatives.Ethereum NuGet package:

private static Func<AuthenticationToken, string, bool> VerifySignature()
{
    return (token, message) =>
    {
        var verifyFunction = ProDerivatives.Ethereum.Signer.VerifySignature();
        return verifyFunction(token.Signature, token.PublicKey, message);
    };
}

Optionally, also add bearer token validation with IdentityServer4. Simply specify authority and API name (aka audience):

services.AddAuthentication(AsymmetricAuthenticationDefaults.AuthenticationScheme)
    .AddAsymmetricAuthentication(options =>
    {
        options.SignatureTokenRetriever = TokenRetrieval.FromAuthenticationHeader();
        options.SignatureValidator = VerifySignature();
    })
    .AddIdentityServerAuthentication(options =>
    {
        options.Authority = Configuration["Settings:Authority"];

        if (Configuration["ASPNETCORE_ENVIRONMENT"] == "Development")
            options.RequireHttpsMetadata = false;

        options.ApiName = "ProDerivatives";
    });

If bearer tokens are used, then subject must match public key of signature.

The asymmetric authentication handler expects an authentication token in the header with following properties:

public class AuthenticationToken
{
    public string Signature { get; set; }
    public string PublicKey { get; set; }
    public long Nonce { get; set; }
    public DateTime Timestamp { get; set; }
}

In your web client, sign all requests to the web API by adding an authentication token to the request header. One simple way to create Ethereum signatures is to use the MetaMask browser plugin. Simply retrieve the connected account (i.e. Ethereum public key) and signatures from the injected ethereum object:

import { ethers } from "ethers"

class EthereumAuthentication {
    constructor() {
        if (typeof ethereum !== 'undefined') {
            this._ethereum = window.ethereum
            var provider = new ethers.providers.Web3Provider(this._ethereum)
            this._signer = provider.getSigner()
        }
    }

    getTicks() {
        const d = new Date()
        const n = d.getTime()
        return n
    }

    async getAuthenticationHeader(action, url) {
        const nonce = this.getTicks()
        let message = `${nonce}|${action}|/${url}|`
        const signerAddress = (await this._ethereum.request({ method: 'eth_accounts' }))[0]
        const signature = await this._signer.signMessage(message)
        const token = {
            PublicKey: signerAddress,
            Signature: signature,
            Timestamp: new Date().toISOString(),
            Nonce: nonce
        }
        const headers = {
            AsymmetricAuthentication: JSON.stringify(token)
        }
        return headers
    }
}


export default {
    EthereumAuthentication
}

Finally, call API endpoints like so:

var header = await auth.getAuthenticationHeader("GET", path)
var result = (await this.$http.get(`${appBasePath}/${path}`, { headers: header })).body.value

If you would like to use authorization tokens in addition to signatures, then also add the bearer token obtained from IdenityServer to each request. Interceptors are a convenient way to set request headers:

Vue.http.interceptors.push((request, next) => {
  const token = localStorage.getItem('user-token')
  if (token) {
    request.headers.set('Authorization', `Bearer ${token}`)
  }
  next()
})

Now all calls to the web API are authenticated and authorized using cryptographic signatures and bearer tokens, respectively.

Finally, in the VerifySignature method of the API server, it is highly recommended to also check that the nonce of the authentication token has been incremented in order to prevent replay attacks.

Guaranteed income for crypto miners

Cryptocurrency miners can use blockchain-based forward contracts to convert their highly variable profit and loss profile into a fixed income stream with assured rentability of their mining assets.

Miners purchase or rent hardware mining units against payment of fiat currency. This upfront investment will only be recovered if the mining output generated by the assets is sufficiently high. While the mined quantity of crypto network tokens can be predicted based on hashpower and difficulty level, the fiat price of those tokens is uncertain and very volatile.

Let's assume a miner pays a cloud mining provider $1,100 per month for a certain amount of hashpower that is expected to yield 5 ETH until the end of the current month. The return of this investment will be determined by the ETH market price at the end of the month when the mining output is sold. For example, over a price range from $200 to $300 per ether, the rentability varies from -9 % to 36 % but the ETH price could go much lower or higher.

ETH Price	Revenue	Return
$ 200	$ 1,000	-9%
$ 210	$ 1,050	-5%
$ 220	$ 1,100	0%
$ 230	$ 1,150	5%
$ 240	$ 1,200	9%
$ 250	$ 1,250	14%
$ 260	$ 1,300	18%
$ 270	$ 1,350	23%
$ 280	$ 1,400	27%
$ 290	$ 1,450	32%
$ 300	$ 1,500	36%

If the miner is unwilling to take the risk of making low or negative returns, they could enter into a forward contract at the same time the mining hardware is rented to lock in a rate of return that will not be affected by adverse price movements over the mining period.

For example, if the forward price locks in a price of $260 per ether then the miner will earn a return of 18 % irrespective of the final price of ether.

ETH Price	Revenue	Forward Payoff	Return
$ 200	$ 1,000	$ 300	18%
$ 210	$ 1,050	$ 250	18%
$ 220	$ 1,100	$ 200	18%
$ 230	$ 1,150	$ 150	18%
$ 240	$ 1,200	$ 100	18%
$ 250	$ 1,250	$ 50	18%
$ 260	$ 1,300	$ -	18%
$ 270	$ 1,350	$ (50)	18%
$ 280	$ 1,400	$ (100)	18%
$ 290	$ 1,450	$ (150)	18%
$ 300	$ 1,500	$ (200)	18%

Moreover, if the rate of return provided by the forward contract is insufficient then the miner has the option to forego the investment or choose a different asset to mine.

How to use ProDerivatives contracts to lock in a rate of return

For the scenario described above, the miner would choose the USD:ETH contract that expires at the end of the mining period. The contract is priced in ETH/USD and so the inverse is the ETH price in dollars that can be locked in, e.g. 0.004 ETH/USD -> 1/0.004 -> 250 USD/ETH which would guarantee a return of 14 % . If the return is acceptable then the miner only needs to decide the amount to cover in the trade. This "notional" amount is simply the amount of ether expected to by mined divided by the forward price, i.e. 5 ETH / 0.004 ETH/USD = $1,250. Therefore, the miner submits a Buy (Long) order for USD 1,250 at 0.004 ETH/USD.

At expiration, the forward contract settlement in ether will make up the difference so that the resulting balance of mining output +/- forward payoff equals $1,250 when sold at the final ETH price.

Fixed-Value Payments

A long position in a fiat-currency forward guarantees the purchase of a certain amount of the underlying (e.g. USD) at a given price (e.g. ETH/USD) when the contract expires. Being assigned an amount of underlying is equivalent to receiving the underlying from the sender. For example, if account A transfers 20 dollars of notional to account B then account B will be credited with an amount of ether that is sufficient to purchase 20 dollars at the market price when the contract expires. The transaction will always be worth 20 dollars regardless of what happens to the ether price of the dollar. It could be compared to the owner of account A handing a cheque for 20 dollars to the owner of account B with the condition that the cheque can only be cashed at the contract expiration date.

The "Notional Transfer" feature of accounts contracts make transactions of this type trivial. It can be used to effect fixed-value payments in exchange for goods and services.

Process

Let's assume a buyer wishes to make payments over the coming month with an expected budget of $100.

1. First, the buyer sells $100 at an exchange and purchases ether which is deposited into an account contract.

2. Using the account contract, the buyer enters in a long USD:ETH forward contract that expires in one month. The underlying notional is the amount of ether obtained in step 1 divided by the forward price. Note, the amount of notional could be more than $100.

3. A first purchase worth $20 is made and 20 dollars notional amount are debited from the buyer's contract position and credited to the vendor's position.

4. Vendor delivers purchased item or service.

5. Both the buyer and vendor can make additional purchases with the (remaining) notional amount.

Assuming no additional purchases are made, the vendor will be credited with an amount of ether equal to the notional ($20) times the current market price of the dollar. This amount is then transferred to an exchange where it is used to buy 20 dollars at the market price. The buyer will be able to retrieve the collateral that still remains in the account contract which will be equivalent in value to the remaining notional amount ($80) at the current market price.

Example

At the beginning of the period, the market price (spot price) of dollars is 0.004 ETH/USD. For simplicity, the forward price for settlement in one month is also assumed to be 0.004 ETH/USD (could be equal, higher or lower than the spot price).

100 dollars are exchanged for 0.4 ETH which are used to collateralize a long position of USD 100.

Spot and Forward Price	0.004	ETH/USD
Notional	100	USD
Collateral	0.4	ETH

A payment of $20 is made sometime during the lifetime of the forward contract.

When the contract expires, the new spot price could be equal, higher or lower than the forward price of 0.004 ETH/USD.

Let's look at each case in turn

Market Price at Expiration: 0.004 ETH/USD

	Position [USD]	Profit/Loss [ETH]	Account Value [ETH]	Account Value [USD]
Buyer	80.00	-0.08	0.32	80.00
Vendor	20.00	0.08	0.08	20.00
Short Cpty	-100.00	0.00

Short counterparties are account holders on the other side of the trade when the buyer established the initial long position of $100.

Market Price at Expiration: 0.005 ETH/USD

	Position [USD]	Profit/Loss [ETH]	Account Value [ETH]	Account Value [USD]
Buyer	80.00	0.00	0.40	80.00
Vendor	20.00	0.10	0.10	20.00
Short Cpty	-100.00	-0.10

Market Price at Expiration: 0.003 ETH/USD

	Position [USD]	Profit/Loss [ETH]	Account Value [ETH]	Account Value [USD]
Buyer	80.00	-0.16	0.24	80.00
Vendor	20.00	0.06	0.06	20.00
Short Cpty	-100.00	0.10

In all cases, the final account value for both buyer and vendor is equal to the amount of notional received or retained times the market price (spot price) at expiration time, allowing conversion back to the appropriate amount of dollars.

A dumb way to test a smart contract

From June 6 to June 14 2019, I ran a user test of a decentralized forward contract on the "Ropsten" network. Ropsten is an Ethereum network where the network token "ether" can be obtained for free and therefore has no financial value.

The test was executed in the form of a trading contest with a total of 3 ETH (approximately $800 or C$1,000 at the time of payment) in prize money. Since trading took place on a test network, a separate financial reward was needed to encourage users to trade the contract. The lack of economic consequences of losing test ether also meant that unrealistic trading behaviour was bound to occur, the most obvious being the execution of trades at inflated or deflated prices between accounts controlled by the same user. Since there was no way to effectively prevent it, the use of multiple accounts to increase chances of winning was explicitly allowed.

There was a positive to this weakness. Testers attempting to win the contest with this strategy could be expected to create a greater number of accounts and trades then they normally would, putting the whole system under a decent amount of "stress" even with relatively few actual individuals involved.

However, there was an alternative scenario. I included a feature in the smart contract which had the power to completely change the way to win the contest if discovered by the testers. Very few trades would be entered but I would have confidence that testers reviewed the contract source code and found an exploit I knew about. If no other exploit would be found during the test, I could argue that the contract was safe at the $1,000 level.

Before I describe what happened, let us look at some stats.

I posted about the contest on Reddit and Gitcoin. All traffic to the contest instructions on this website came from Gitcoin.

Website traffic

Number of unique visitors:	109
Visits:	139
Number of Page Views:	317
Countries:	34	(from all continents except Antarctica)

Looking at user flow, over 90% of visitors only accessed information directly related to the contest and did not explore the rest of the website.

Test stats

Number of testers:	9
Number of test accounts (peak):	16
Number of trades entered:	251
Total amount traded:	$163,185
Total test ether supplied (peak):	165 ETH
Winners:	2	(one tester took 1st and 3rd place)

Communication with testers and feedback

Communication with testers happened primarily on the test's page on Github.

Two testers asked three questions primarily related to the test object.

One of the questions concerned the possibility of gaming the system by trading between own accounts and suggested that it should be "restricted." I replied that it was “perfectly legal.”

One tester reported a user interface bug and also followed up with an email. I exchanged 11 messages with this user and I had the impression that they were providing honest feedback and were willing to work through issues. They mentioned that the application provided a "low user experience" and "high technical difficulty" and offered to provide further recommendations and feedback for a fee.

Two testers completed the test survey which included questions about bugs and recommendations. One user stated that “lots” of (unspecified) improvements could be made. Another user claimed to have found other bugs and exploits in addition to trading between own accounts, but did not give details. The tester did, however, send an email offering help with testing and fixing the contract “with or without a bounty.” Both testers stated that the documentation “could have been better.”

Test History

Below OHLC chart shows the price of the underlying (Ethereum ETH in US dollars) during the test period. The frequent trend changes and volatility created variability in the relative day-to-day performance of the competitors. This was probably beneficial to the test as it may have encouraged continued engagement, i.e. trading, by the testers.

The next chart shows volumes (bottom section) and prices of the underlying traded via the derivative contract that was the focus of the test. The derivative used the inverse of the underlying price (e.g. $250 per ETH converts to 1/250 = 0.004 ETH per $).

The derivative price chart shows a relatively smooth line and a clear negative correlation with the underlying price during the first 5 days. Traded volumes were low.

On June the 9th, the question of trading between own accounts was raised on the Github issue page and the first evidence of this "strategy" being used occurred on June 10. The pronounced kinks and trend deviations in the second half of the graph are caused by implementations of the strategy. The "flash crash" on June 11 could be a failed attempt, but judging by the accounts and amounts involved, it was a data entry error.

Although it is impossible to be absolutely certain, according to my analysis only 3 test participants made use of this method to increase their chances of winning. Furthermore, only 24 out of 251 trades match this pattern. Volumes traded on these occasions were high and the strategy was the determining factor of the final ranking. 23 out of the 24 "strategy trades" can be attributed to the two winners.

Below graph illustrates a well-executed instance of the pattern. The account selected to be the winner buys at a low price from another account controlled by the same tester and then an opposing trade is entered shortly afterwards at a considerably higher price. The notional amount traded in this example was $10,000. The process is repeated a few times and the buy-low-sell-high account is left with a profit.

Assuming that all testers had some interest in winning the contest (as it was the only way to gain financially from participating) it is noteworthy that the strategy was not used very aggressively, i.e. used only by a small number of testers, on a few occasions, and with moderate profit margins.

Trading activity was uneven. Almost 60% of all trades occurred over the span of two days from June 11 to June 12.

Period	Trades in period
June 6 - 10	61
June 11	49
June 12	96
June 13	24
June 14	21

The "special" feature of the contract mentioned in the introduction would have made a surprise win possible if exploited just before the test ended (so as not to alert other participants), but it was not used.

Personal Conclusions and Recommendations

Offering bounties can attract a significant number of motivated individuals to participate with testing or other aspects of a software development project.

It can be expected that participants will adapt and optimize their behavior according to the incentive structure offered and not exert much effort unless it is directly rewarded. Testers may not even report bugs unless properly incentivized.

Care must be taken to offer sufficiently high bounties for every meaningful contribution. As such, it is more suitable for problems that are well-defined.

1st Decentralized Derivative Trading Contest: USD:ETH/M9 (managed) on Ropsten

The contest ends Friday, June 14 at 16:30 UTC and remains open to new participants until the end.

Rankings

Current and final rankings are published on the Github issue page. Please watch the issue to receive updates.

Quick Start Guide

1. Sign up for the contest on Gitcoin

2. Start local Ropsten node with unlocked account or log into MetaMask and open a Ropsten wallet

3. Obtain test ether from https://faucet.ropsten.be if required

4. Go to https://prod.proderivatives.com click on "ETH" and wait until "Ropsten" appears in the title

5. Open "Accounts" and deploy a new account contract

6. Fund your account contract with test ether but leave some in your wallet to pay network transaction fees

7. Add new derivative and select "USD:ETH/M9"

8. Click the derivative contract and in the Operations menu, select "Pay Fee"

9. Wait until the fee has been paid and your account has been verified (checkmark appears in "Verified" column) - this may take a few minutes

10. Go to "Trading", select "USD:ETH/M9" and submit your first order

One trade could be enough to secure the bounty of 1.5 ETH. The second and third-best accounts will be awarded 1 ETH and 0.5 ETH, respectively. It is okay to use multiple account contracts for trading, so if the top 3 accounts are controlled by a single trader such as yourself, you will receive 3 ETH. Don’t miss out!

For more information, please visit https://proderivatives.com/prod or ask a question on the support chat channel (click “Support” button in the trading app).

Introduction

For this contest you will assume the role of a derivatives trader. Your task is to execute profitable trades with other users to build up a position that is "in-the-money" when the contract expires. The trader with the highest in-the-money position at contract expiration wins.

Basics

The derivative used in this contest is a Forward contract (aka Futures contract). The value of a forward is a linear function of the price of the underlying asset. The profit or loss you make on a trade is the difference of the underlying price at termination and the price at which you executed the trade times the notional amount. If you entered into a long position then you will gain if the price difference is positive and lose if it is negative. For a short position it is the exact reverse.

For more information about contract specifics, please visit https://proderivatives.com/contracts/usdethm9

Getting Started

In order to interact with the derivative contract, you first need to create your own instance of an approved account contract. This is a smart contract owned and operated by you. Its main functions are to provide an interface to the derivative contract and to hold the collateral for your positions. The amount of ether you deposit into your account contract is the maximum you can lose. Once the derivative contract has been settled, you can transfer the remaining balance out of your account contract and destroy it or continue to use it with other derivatives if you wish.

To review the source code of the account contract specification, please visit https://raw.githubusercontent.com/ProDerivatives/EthereumContracts/master/contracts/Account.sol

To read more about the mechanics of the contracts, please visit https://proderivatives.com/contracts

In order to participate in the contest, you need to deploy the account contract to the Ropsten test network and fund it with test ether. You can do this with your preferred tool for interacting with Ethereum-based smart contracts such as Truffle or Remix. For your convenience, a purpose-built web application is available which provides a GUI to perform most tasks. The web application can be launched from https://proderivatives.com/prod

The web application directly interacts with the blockchain using MetaMask or your local Ropsten node just as Truffle and Remix would.

The Contest

All verified account contracts registered with derivative deployed at 0x7e49ec8389335a6280fbd411d5ac704e646632a9 on the Ropsten network participate in the contest. Once registered, you can trade as much as you want to increase your chances of winning.

The contract used for the contest is a "managed forward" meaning that ProDerivatives will perform operational tasks for the benefit of all users:

• Daily valuation between 16:30 and 17:00 UTC

• Monitoring of collateral positions

• Close-out of positions in default

• Settlement within 24h after expiration

It is your responsibility to ensure that your account contracts holds sufficient collateral to cover your position and open orders. Should you fail to do that, ProDerivatives will attempt to reduce or close your position by posting offsetting orders at the price calculated by the derivative contract. The price includes a spread against your favor designed to attract traders willing to take over your position. It is more beneficial to you if you close your position yourself rather then letting the contract do it for you. On the other hand, you may be able to enter into profitable positions if you manage to pick up a close-out order from a defaulting counterparty.

Hint: The collateral requirement for a long position cannot exceed the position amount (notional * execution price). If you deposit this much collateral then there is no need to monitor the account.

After contract expiration and settlement, accounts will be ranked by the settlement amount they received.

The owners of the account contracts that receive the first, second, and third highest settlement amount are awarded ETH 1.5, 1, and 0.5, respectively. Winners will be asked to provide an Ethereum Mainnet address to receive the award.

Following account contracts are market-making and test accounts owned by ProDerivatives and will be ignored in the ranking:

0xb2fc43c41ddf1f80a7b93a819e97c9caef2ab7de,

0xcb53b4018088b31eb1b214f453be1754786e8d42.

Contact

Please email info@proderivatives.com with any questions or concerns and keep an eye an this blog posts for updates during the contest.

Additional contests featuring more complex contracts with higher payouts may be coming in the future.

Time and Volume Weighted Price Index

The value of a derivative depends on the price of its underlying. ProDerivatives publishes prices of underlyings in the form of a price index that is updated daily. The index is calculated from market prices and typically taken from actual trades reported by supported trading venues. Rather than simply using the last trade of the trading cycle, ProDerivatives samples trades over the complete cycle (i.e. over one day) and weighs each price by the relative point in time when the trade occurred and the volume of the trade. This method is called time and volume weighted index pricing.

Advantages:

• Every trade of the trading cycle contributes to the index making it a more accurate representation of the trading cycle.

• Index value does not take on the extreme high or low price observed during the trading cycle.

• It is more difficult to game the index by placing untypical trades at the end of the trading cycle.

Minimizing Data Storage Cost on the Ethereum Network

Update...

Scroll all the way to the bottom of this post to see how the storage cost has evolved over the past 12 months.

Introduction

One of the key benefits of the Ethereum network is decentralized, tamper-proof data storage. Attempts have been made to calculate the cost of storage, for example Hess, 2016 and Omaar, 2017 estimate the cost of storing 1 GB worth of data at $76,000 and $4,672,500, respectively. The amounts are different because the gas price and especially the Ether price changed considerably between February 2016 and July 2017. ¹ The difference would have been even larger if the calculations had been based on the same amount of gas. Omaar estimated that the operation would require 625 billion gas, while Hess calculated an amount of 700 billion. ² Had Omaar used the higher amount then July 2017 storage cost would increase to $5,236,645 per GB.

Gas usage depends on the set of instructions executed by a transaction. The amount of gas consumed by each instruction is defined in the Ethereum protocol and does not change unless the protocol changes (see Wood). Therefore, gas usage does not depend on network or market conditions and identical transactions will use the same amount of gas regardless of when they are executed. A difference of 75 billion gas for effectively the same transaction is therefore difficult to explain.

In order to obtain the correct amount of gas it is necessary to map gas usage as a function of the number of bytes sent to and persisted on the Ethereum network.

Gas usage as a function of bytes sent for storage

In order to calculate the exact amount of gas required for sending and storing a given number of non-zero bytes, first obtain the amount of gas required to write the initial 32 bytes. This amount can vary depending on the specific contract implementation and data structure used. ³

Between 32 and the required number of bytes:

• Add 64 gas for every byte.

• Add 134 gas after each block of 32 bytes.

• Add 20,070 gas (see below) after each block of 32 bytes.

• Add 64 gas from byte 256, skip every integer multiple of 256.

• Add 64 gas from byte 65,792, skip for 256 bytes every integer multiple of 65,536.

• After byte 544 add 1 gas according to sequence shown in the appendix. One additional sequence is applied every 32 * 256 bytes.

A sample implementation of the gas usage function can be found here.

Only the 20,070 gas after each block of 32 bytes is directly attributable to storage. All other items apply even if no data is persisted on the blockchain.

The compounding sequences (last item) cause the gas usage function to go superlinear - usage of gas increases at a growing rate. In turn this means that average gas usage (total gas used / number of bytes) first falls as initial transaction gas usage is distributed over a larger number of bytes and then rises again because marginal gas usage (gas used by last byte added) increases. The number of bytes at which average gas usage is minimized is the cutoff point at which large transactions should be split up, i.e. once the number of bytes to be written is large enough, it is cheaper to send multiple smaller transactions instead of one big one.

The theoretical optimal transaction size is approximately 150 KB⁴ and requires a little over 105 million gas. A 1 GB transaction should be split into 6,587 transactions with a total usage of 695 billion gas. This amount is reasonably close to the 700 billion gas underlying the estimate provided by Hess. Since 695 billion gas is the minimal gas amount, 625 billion gas is too low.

However, 105 million gas far exceeds the block gas limit at the time of writing meaning that 695 billion gas is currently not achievable. With a block gas limit of 8 million, the optimal transaction size decreases to 11,424 bytes⁵ and total gas usage for storing 1 GB increases to 700 billion gas. This amount is the currently achievable minimum.

Multiplying 700 billion gas with the current gas price and Ether price yields the dollar cost of sending and storing 1 GB.

With a gas price of 12 Gwei and 200 $/ETH at the time of writing in early November 2018, it costs $1.68 million to store 1 GB on the Ethereum blockchain.

Footnotes

1. 50 Gwei per unit of gas and $2.17 per ETH in February 2016 vs 28 Gwei per unit of gas and $267 per ETH in July 2017

2. 700 billion gas vs 625 billion. Gas Usage [units of gas] = Cost [$] / ( Gas Price [Ether / gas] * Network Token Price [$ / Ether]).

3. 65,000 gas is a representative amount.

4. Specifically, gas usage is minimized at 151,808 bytes and requires at least 105,585,348 gas. The contract implementation or data structure used should not make much of a difference at this size.

5. Since storage is allocated in blocks of 32 bytes, it is efficient to size transactions in integer multiples of 32, e.g. 1024 bytes instead of 1000 bytes.

References

Omaar, Jamila. 2017. Forever isn’t free: The cost of storage of blockchain on a blockchain database.  https://medium.com/ipdb-blog/forever-isnt-free-the-cost-of-storage-on-a-blockchain-database-59003f63e01

Hess, Tjaden. 2016. Answer to: What is the cost to store 1KB, 10KB, 100KB worth of data into the ethereum blockchain?

https://ethereum.stackexchange.com/questions/872/what-is-the-cost-to-store-1kb-10kb-100kb-worth-of-data-into-the-ethereum-block

Wood, Gavin. Ethereum: A secure decentralised generalised transaction ledger. http://gavwood.com/paper.pdf

Appendix

Below sequence shows blocks of bytes which trigger one additional unit of gas to be added to the total cost of gas. The sequence starts at byte 544 and continues forever and a new sequence is added every 8192 bytes. This causes average gas cost per byte to rise with increasing transaction storage requirement.

Add 1 gas after bytes	Iterations
288	1
256	1
192	1
160	2
128	5
96	6
64	1
96	2
64	1
96	1
64	2
96	1
64	15
32	1
64	3
32	1
64	2
32	1
64	1
32	1
64	2
32	1
64	1
32	1
64	1
32	1
64	1
32	1
64	1
32	1
64	1
32	1
64	1
32	2
64	1
32	2
64	1
32	2
64	1
32	2
64	1
32	2
64	1
32	3
64	1
32	3
64	1
32	4
64	1
32	6
64	1
32	7
64	1
32	Forever

Blockchain Networks as Technical Infrastructure

One of the key benefits of distributed ledger technologies is decentralized, tamper-proof data storage. A blockchain network such as Ethereum or Bitcoin can take data transmitted via transactions, add it to a block and persist copies of it on every node. As long as the network is alive the data can be recovered and used independently of the action or inaction of any individual agent including the original sender of the data, who would normally have to incur perpetual expenses for hosting and securing the data.

The network charges a one-time fee for providing this service for its remaining lifetime to cover energy, capital and labor costs over a timespan of unknown length. The fee is charged at transaction time and in the case of Ethereum is calculated as:

Gas Usage [units of gas] *

Gas Cost [Ether / gas] *

Network Token Price [$ / Ether]

Transaction gas usage is a function of bandwidth, storage and compute resources consumed by the transaction and can be calculated before submitting the transaction. It is a nominal amount that does not represent its real resource cost. An identical transaction will always use the same amount of gas regardless of the state of the network at the time it is submitted unless the network protocol itself is changed between transactions.

Gas cost is chosen by the user. The higher the amount offered by the user the faster the transaction will be included in a block. The relationship can be estimated from network statistics.

The fiat currency price of the network token (in this case Ether) is a market price that is determined on exchanges where network tokens are traded for fiat currency.

All three factors combined should (but not necessarily do) represent the real resource cost incurred by the transaction discounted to the present. The resulting value is transferred from the user to the network (miners in the case of Ethereum and Bitcoin).

It is reasonable to expect that networks will compete for users based on this resource cost, i.e. if it is cheaper to store data of a certain size on network A compared to an otherwise identical network B then network A should acquire more users interested in this particular feature, for example software developers who require self-updating local storage for building temporarily disconnected applications.