November 2012 summaries

Modelling the elastic and barrier properties of the skin

The topmost layer of the skin forms an effective barrier to water loss and protects against the adsorption of substances from outside the body. Proteins found in this top layer of the skin, called keratins, assemble hierarchically into filaments (dimers → tetramers → higher-order assemblies → protofilaments). These fibres are thought to play an active role in underpinning the barrier properties, appearance, and elasticity of skin. Little is known about how keratin fibrils confer these properties – chiefly because it is experimentally very challenging to directly obtain the entire atomic-level structure of the smallest sub-units of this protein assembly (the keratin dimer and tetramer). Molecular dynamic (MD) simulation provides a route to proposing and generating credible structural models of the keratin sub-unit organisation.

The aim of this project was to establish the first credible structural models of the keratin tetramer, a key sub-unit of the keratin fibril, and thus enable the subsequent utilization of these models in making the first definitive links between molecular structure and properties (eg mechanical response) of these keratin sub-units. The specific objectives were to:

(1) Generate approximate structural models of the keratin tetramer sub-unit, derived from atomistic simulation using implicit solvent.

(2) Refine these implicit solvent tetramer structures by using the structures from (1) in liquid water (described atomistically), and compare these against experimental evidence.

(3) Characterize the solvent structuring around key sites of the keratin tetramer sub-unit, and use these data to propose the first links between keratin (and solvent) structure and mechanical properties of keratin.

Using atomistic simulations of various combinations of keratin dimers, we have constructed and investigated the key interfaces present in the keratin protofilament and filament. With large-scale MD simulations we have investigated the four principal tetramer arrangements present in the keratin fibre (denoted as A11, A22, A12, and ACN in the literature). To our knowledge, the A22, A12 and ACN arrangements have never been computationally explored. Our simulations hence represent the first time such structures have been investigated and we have determined key details on the critical role played by the head, tail and rod domains in these interactions.

Turbulent mass transfer at high Schmidt number

Turbulent mass transfer is of importance for several problems in the geophysical and engineering sciences. Examples are corrosion processes and the absorption of nutrients and dissolved oxygen by biofilms and micro-organisms in the sediment. The common factor in these processes is that these solutes have a very low molecular diffusivity and consequently their Schmidt number is very high. The project had two main aims. The first aim was to determine the effect of boundary conditions on the mass transfer coefficient, as 1) Taylor series expansion suggest they will be of significant influence [1]; and 2) there are strong indications that at high Schmidt number, the analogy between momentum and mass transfer breaks down because the concentration boundary layers become extremely thin at high Schmidt numbers. The second aim was to collect statistics required for the development of accurate turbulence models for the prediction of high Schmidt number turbulent mass transfer. We performed direct numerical simulation (DNS) of turbulent plane channel with six passive scalars simultaneously for Schmidt numbers up to 100. Particularly noteworthy is that we simulated all six passive scalars simultaneously which allowed us to compare the Schmidt-number dependent response to an exactly identical flow evolution. The simulation of high Schmidt number mass transfer was non-trivial, as the simulations require very fine grids whilst the high Schmidt numbers required implicit treatment of the viscous terms to avoid time-step restrictions (ironically the unity Schmidt number scalars created the time-step limitation).

The influence of the different boundary condition types appears to be significant for unity Schmidt numbers predominantly, and at high Schmidt numbers there is little difference between the boundary conditions. We are currently looking into the reason why the leading order term for the Neumann boundary condition is forced to zero as the Schmidt number increases.

The DNS data provides valuable information for turbulence modellers, as we have collected detailed budgets of the turbulent fluxes of mass and momentum. These are critical in the development of accurate turbulence models. Indeed, the data clearly shows that the current turbulence models are unable to capture even remotely, high Schmidt number turbulent mass transfer.

Cloverleaf: preparing hydrodynamics codes for exascale

This project concerned the benchmarking and performance evaluation of CloverLeaf, an application proxy developed by academia and industry. This was developed in order to understand how 2 and 3 dimensional Eulerian structured mesh hydrocodes could be improved in order for their performance to be scaled acceptably to Exascale levels on future high performance computing (HPC) architectures. The objectives of, and research outcomes arising from, this study were:

(1) to evaluate the performance of three versions of CloverLeaf: flat Message Passing Interface (MPI), hybrid (MPI+Open Multi Processing (OpenMP)) and Co-array Fortran (CAF) to determine which of these programming models exhibits the most promising scalability. As part of this work we conducted scaling experiments of multiple code variants, each based on one of these three programming models, for a range of different problem sizes and job scales. In both strong- and weak-scaling scenarios, up to the maximum job size available on the machine (2048 nodes).

(2) to gather data from these runs on compute/communication breakdown and overlap so that potential bottlenecks at Exascale can be anticipated and strategies can be investigated to mitigate for these effects. We used the data gathered from the aforementioned runs to investigate the ratio of computation to communication within the application for each problem size and execution scale examined. Additionally, we also examined the effectiveness of the system at overlapping communication and computation as a strategy for improving performance and scalability on HPC systems of this class.

(3) to investigate different optimisation techniques to improve the scalability of CloverLeaf, which will allow for the most performant execution at scale. We investigated a range of candidate techniques for improving the scalability of CloverLeaf and assessed their utility at a range of different scales. For the MPI based derivatives these included: hybridising the code using OpenMP; reordering the MPI ranks to improve the mapping between the application and the underlying network topology; diagonal communications to reduce synchronisation; message aggregation; pre-posting MPI receives; utilising sequential memory and MPI datatypes; actively checking for MPI message arrivals; and overlapping communications with computation. Additionally, for the CAF based derivatives we examined different synchronisation approaches; using one-sided “gets” rather than “puts”; and communication buffer-based exchanges instead of the more usual arraysection based approach.

(4) to use the outputs from (1)-(3) to inform language, algorithmic and coding choices in our search for an Exascale–ready 3D Eulerian structured mesh hydrocode.

The most successful optimisations from our work have been incorporated into the version 1.1 release of the CloverLeaf proxy-application. Additionally, we have also published our results (see below) to enable other researchers to benefit from our work.

(5) to publish this work and to make the techniques and codes available to the wider research community.

Atomic data for fusion diagnostics

We requested High End Computing Terascale Resources (HECToR) time to commence preliminary investigations on specific ions of the heavy element tungsten for applications in fusion and in particular International Thermonuclear Experimental Reactor (ITER) diagnostics. The role of tungsten as a suitable thermally resistant plasma facing wall component for the diverters of fusion machines has stimulated fresh studies of the atomic physics of tungsten. The work proposed here focuses on W XLV, a spectral line emitter in the European Fusion Development Agreement (EFDA)-Joint European Torus (JET) experiment soft xray regime. A significant amount of initial work on the atomic structure of W XLV has already been performed by the applicants and we are now in a position to progress to the collision problem. A 326-state structure description is optimally necessary for the accurate representation of the primary observable transitions – a formidable calculation for such a heavy element. We wish to perform some less complicated feasibility studies to test the limit of current capability for the R-matrix method. A fully relativistic Defra Antimicrobial Resistance Co-ordination (DARC) treatment of a 326 level W XLV model would be impossible, even on current supercomputing resources. We would like to test how the semi-relativistic PRMAT and Breit-Pauli RMATRX I codes perform for these heavier ions by utilizing an adjusted model potential in the optimization of the orbital parameters. For heavier ions, relativistic effects contribute not only to the energies but also to the radial distribution of the orbitals. This relativistic adjustment of the orbitals can be achieved by including the mass-correction and Darwin BP terms in the optimization process. Extensive atomic structure calculations for W XLV have shown that a very good target model can be established using this procedure.

To compute the collision cross sections the parallel R-matrix codes, PRMAT, PFARM and PSTGF, as well as the parallel variant of the Breit-Pauli semi-relativistic codes RMATRX I, will be utilized. These codes have been developed to run efficiently on the national supercomputing facilities (Cray T3E and High Performance Computer (HPCx)) since 2001, and are currently running on HECToR. The proposed calculations on the computationally challenging tungsten ions depend entirely on our continued access to state-of-the-art massively parallel computing architectures. Our main objective is to produce highly accurate electron impact excitation cross section data for high ionization stage ions of tungsten.

We have so far managed to construct optimum orbital parameters to describe the 326-state model described above. A complete set of radiative data to include energy levels, oscillator strengths and transition probabilities have been computed. These data compare favourably with observations and other theoretical evaluations. From January to June 2013, we commenced the running of the collisional calculations on HECToR, starting with some small feasibility models. We managed to run all our models through the internal (PRMAT) and external (PSTGF) regions to compute the electron-impact excitation collision strengths and effective collision strengths. The 326-state optimum model posed no difficulties and we succeeded in completing the evaluations for this large calculation. We used up all the allocated resources provided to us.

Mesoscale modelling of offshore wind

The overall aim of this project was to assess the use of the Weather Research and Forecasting (WRF) mesoscale model for predicting offshore wind resource. The objectives were:

  • To assess the accuracy of wind speed and direction prediction for two offshore sites (Scoby Sands and Shell Flats) where mast data were available;
  • To determine how the accuracy varies under different climatic conditions;
  • To assess different problem-based learning (PBL) schemes to see which performs best;
  • To investigate the use of ensembles to improve prediction accuracy;
  • To assess model accuracy in predicting atmospheric stability conditions at the Shell Flats offshore site;
  • To predict the expected resource variation at a hypothetical Round 3 wind farm site in Dogger Bank as part of the EPSRC Supergen Wind consortium project.

The major research outcomes are:

  • Model accuracy depends on distance from the shore: for the near coastal site (Scroby Sands) prediction accuracy was not as good at the site at the further offshore Shell Flats (~15km from the coast);
  • Using a filter to remove high frequency variations (higher than can be resolved by the model) improves model accuracy;
  • Root mean squared wind speed prediction error on the basis of six-hourly filtered data was found to be 1.7m/s at Shell Flats. On the basis of three-hourly filtered data at Scroby Sands, it was found to be 1.9m/s with an optimised set-up
  • Optimised model set-up give a correlation coefficient between measured and modelled wind speeds of 0.90 for Shell Flats and 0.72 for Scroby Sands;
  • Good agreement was seen between modelled and measured wind roses at Shell Flats;
  • There is reasonable agreement between measured and modelled stability statistics at Shell Flats, albeit the model shows a bias to more stable conditions. This needs more investigation to assess a larger range of PBL schemes and validation at further sites;
  • Investigation of the hypothetical Round 3 wind farm site shows some periods of significant variation in wind conditions across the site (over ~20km), eg large >50 degree difference in wind direction in some extreme cases.

Aeroengine aeroacoustic interactions

The broad objective of the work was to study the influence of geometry on the aerodynamics and aeroacoustics of jet plumes using Large Eddy Simulation (LES). The area is of extreme environmental importance. The higher the bypass ratio, the more efficient the engine. However, such engines have a very large diameter. Hence, the engine becomes closer to the wing and the question is how the engine flow interacts with the wing? Also, it seems important to assess how all the complexity inside the engine impacts on the jet shear layer development. We observed an intense interaction between the jet and wing and a substantial impact of jet geometry on shear layer development.