Skip to content

Commit 38afb0e

Browse files
aaron-laneaaron-lane
authored
Merge pull request #1 from ogreface/master
Initial Commit
2 parents 82d2619 + 2b94122 commit 38afb0e

37 files changed

+1650
-2
lines changed

‎.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
terraform.tfstate
2+
*.tfstate.*
3+
.terraform
4+
credentials.json
5+
*.iml
6+
.idea
7+

‎.kitchen.yml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Copyright 2019 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
---
16+
driver:
17+
name: "terraform"
18+
command_timeout: 1800
19+
20+
provisioner:
21+
name: "terraform"
22+
23+
platforms:
24+
- name: local
25+
26+
suites:
27+
- name: "pubsub_scheduled"
28+
driver:
29+
name: "terraform"
30+
command_timeout: 1800
31+
root_module_directory: test/fixtures/pubsub_scheduled
32+
verifier:
33+
name: terraform
34+
color: true
35+
systems:
36+
- name: pubsub_scheduled
37+
backend: local
38+
controls:
39+
- gcloud
40+
provisioner:
41+
name: terraform

‎CHANGELOG.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog][keepachangelog-site],
6+
and this project adheres to [Semantic Versioning][semver-site].
7+
8+
9+
## 0.1.0 - 2018-08-08
10+
11+
### Added
12+
13+
- Initial release
14+
15+
[keepachangelog-site]: https://keepachangelog.com/en/1.0.0/
16+
[semver-site]: https://semver.org/spec/v2.0.0.html

‎Gemfile

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Copyright 2019 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
ruby '2.5.3'
16+
17+
source 'https://rubygems.org/' do
18+
gem 'kitchen-terraform', '~> 4.3'
19+
end

‎LICENSE

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
12
Apache License
23
Version 2.0, January 2004
34
http://www.apache.org/licenses/
@@ -198,4 +199,4 @@
198199
distributed under the License is distributed on an "AS IS" BASIS,
199200
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200201
See the License for the specific language governing permissions and
201-
limitations under the License.
202+
limitations under the License.

‎Makefile

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# Copyright 2019 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Make will use bash instead of sh
16+
SHELL := /usr/bin/env bash
17+
18+
# Docker build config variables
19+
CREDENTIALS_PATH ?= /cft/workdir/credentials.json
20+
DOCKER_ORG := gcr.io/cloud-foundation-cicd
21+
DOCKER_TAG_BASE_KITCHEN_TERRAFORM ?= 0.11.11_235.0.0_1.19.1_0.1.10
22+
DOCKER_REPO_BASE_KITCHEN_TERRAFORM := ${DOCKER_ORG}/cft/kitchen-terraform:${DOCKER_TAG_BASE_KITCHEN_TERRAFORM}
23+
24+
# All is the first target in the file so it will get picked up when you just run 'make' on its own
25+
all: check_shell check_python check_golang check_terraform check_docker check_base_files test_check_headers check_headers check_trailing_whitespace generate_docs
26+
27+
# The .PHONY directive tells make that this isn't a real target and so
28+
# the presence of a file named 'check_shell' won't cause this target to stop
29+
# working
30+
.PHONY: check_shell
31+
check_shell:
32+
@source test/make.sh && check_shell
33+
34+
.PHONY: check_python
35+
check_python:
36+
@source test/make.sh && check_python
37+
38+
.PHONY: check_golang
39+
check_golang:
40+
@source test/make.sh && golang
41+
42+
.PHONY: check_terraform
43+
check_terraform:
44+
@source test/make.sh && check_terraform
45+
46+
.PHONY: check_docker
47+
check_docker:
48+
@source test/make.sh && docker
49+
50+
.PHONY: check_base_files
51+
check_base_files:
52+
@source test/make.sh && basefiles
53+
54+
.PHONY: check_trailing_whitespace
55+
check_trailing_whitespace:
56+
@source test/make.sh && check_trailing_whitespace
57+
58+
.PHONY: test_check_headers
59+
test_check_headers:
60+
@echo "Testing the validity of the header check"
61+
@python test/test_verify_boilerplate.py
62+
63+
.PHONY: check_headers
64+
check_headers:
65+
@source test/make.sh && check_headers
66+
67+
# Integration tests
68+
.PHONY: test_integration
69+
test_integration:
70+
./test/ci_integration.sh
71+
72+
.PHONY: generate_docs
73+
generate_docs:
74+
@source test/make.sh && generate_docs
75+
76+
# Versioning
77+
.PHONY: version
78+
version:
79+
@source helpers/version-repo.sh
80+
81+
# Run docker
82+
.PHONY: docker_run
83+
docker_run:
84+
docker run --rm -it \
85+
-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
86+
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
87+
-e TF_VAR_project_id \
88+
-v $(CURDIR):/cft/workdir \
89+
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
90+
/bin/bash
91+
92+
.PHONY: docker_create
93+
docker_create:
94+
docker run --rm -it \
95+
-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
96+
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
97+
-e TF_VAR_project_id \
98+
-v $(CURDIR):/cft/workdir \
99+
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
100+
/bin/bash -c "kitchen create"
101+
102+
.PHONY: docker_converge
103+
docker_converge:
104+
docker run --rm -it \
105+
-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
106+
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
107+
-e TF_VAR_project_id \
108+
-v $(CURDIR):/cft/workdir \
109+
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
110+
/bin/bash -c "kitchen converge && kitchen converge"
111+
112+
.PHONY: docker_verify
113+
docker_verify:
114+
docker run --rm -it \
115+
-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
116+
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
117+
-e TF_VAR_project_id \
118+
-v $(CURDIR):/cft/workdir \
119+
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
120+
/bin/bash -c "kitchen verify"
121+
122+
.PHONY: docker_destroy
123+
docker_destroy:
124+
docker run --rm -it \
125+
-e CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=${CREDENTIALS_PATH} \
126+
-e GOOGLE_APPLICATION_CREDENTIALS=${CREDENTIALS_PATH} \
127+
-e TF_VAR_project_id \
128+
-v $(CURDIR):/cft/workdir \
129+
${DOCKER_REPO_BASE_KITCHEN_TERRAFORM} \
130+
/bin/bash -c "kitchen destroy"
131+
132+
.PHONY: test_integration_docker
133+
test_integration_docker: docker_create docker_converge docker_verify docker_destroy
134+
@echo "Running test-kitchen tests in docker"

‎README.md

Lines changed: 150 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,150 @@
1-
# terraform-google-scheduled-function
1+
# Terraform Google Scheduled Functions Module
2+
This modules makes it easy to set up a scheduled job to trigger events/run functions.
3+
4+
## Usage
5+
You can go to the examples folder, however the usage of the module could be like this in your own main.tf file:
6+
7+
```hcl
8+
module "scheduled-function" {
9+
source = "terraform-google-modules/scheduled-functions/google"
10+
version = "0.1.0"
11+
project_id = "<PROJECT ID>"
12+
job_name="<NAME_OF_JOB>"
13+
schedule="<CRON_SYNTAX_SCHEDULE"
14+
function_entry_point="<NAME_OF_FUNCTION>"
15+
function_source_directory="<DIRECTORY_OF_FUNCTION_SOURCE>"
16+
name="<RESOURCE_NAMES>"
17+
region="<REGION>"
18+
}
19+
```
20+
21+
Then perform the following commands on the root folder:
22+
23+
- `terraform init` to get the plugins
24+
- `terraform plan` to see the infrastructure plan
25+
- `terraform apply` to apply the infrastructure build
26+
- `terraform destroy` to destroy the built infrastructure
27+
28+
[^]: (autogen_docs_start)
29+
30+
## Inputs
31+
32+
| Name | Description | Type | Default | Required |
33+
|------|-------------|:----:|:-----:|:-----:|
34+
| bucket\_name | The name to apply to the bucket. Will default to a string of <project-id>-scheduled-function-XXXX> with XXXX being random characters. | string | `""` | no |
35+
| function\_available\_memory\_mb | The amount of memory in megabytes allotted for the function to use. | string | `"256"` | no |
36+
| function\_description | The description of the function. | string | `"Processes log export events provided through a Pub/Sub topic subscription."` | no |
37+
| function\_entry\_point | The name of a method in the function source which will be invoked when the function is executed. | string | n/a | yes |
38+
| function\_environment\_variables | A set of key/value environment variable pairs to assign to the function. | map | `<map>` | no |
39+
| function\_event\_trigger\_failure\_policy\_retry | A toggle to determine if the function should be retried on failure. | string | `"false"` | no |
40+
| function\_labels | A set of key/value label pairs to assign to the function. | map | `<map>` | no |
41+
| function\_name | The name to apply to the function | string | n/a | yes |
42+
| function\_runtime | The runtime in which the function will be executed. | string | `"nodejs6"` | no |
43+
| function\_source\_archive\_bucket\_labels | A set of key/value label pairs to assign to the function source archive bucket. | map | `<map>` | no |
44+
| function\_source\_directory | The contents of this directory will be archived and used as the function source. | string | n/a | yes |
45+
| function\_timeout\_s | The amount of time in seconds allotted for the execution of the function. | string | `"60"` | no |
46+
| job\_description | Addition text to describet the job | string | `""` | no |
47+
| job\_name | The name of the scheduled job to run | string | n/a | yes |
48+
| job\_schedule | The job frequency, in cron syntax | string | `"*/2 * * * *"` | no |
49+
| message\_data | The data to send in the topic message. | string | `"dGVzdA=="` | no |
50+
| project\_id | The ID of the project where this VPC will be created | string | n/a | yes |
51+
| region | The region in which resources will be applied. | string | n/a | yes |
52+
| topic\_name | Name of pubsub topic connecting the scheduled job and the function | string | `"test-topic"` | no |
53+
54+
## Outputs
55+
56+
| Name | Description |
57+
|------|-------------|
58+
| name | The name of the job created |
59+
60+
[^]: (autogen_docs_end)
61+
62+
## Requirements
63+
### Terraform plugins
64+
- [Terraform](https://www.terraform.io/downloads.html) 0.11.x
65+
- [terraform-provider-google](https://github.com/terraform-providers/terraform-provider-google) plugin v2.1
66+
67+
### App Engine
68+
Note that this module requires App Engine being configured in the specified project/region.
69+
This is because Google Cloud Scheduler is dependent on the project being configured with App Engine.
70+
Refer to the [Google Cloud Scheduler documentation](https://cloud.google.com/scheduler/docs/) for more information on the App Engine dependency.
71+
72+
The recommended way to create projects with App Engine enabled is via the [Project Factory module](https://github.com/terraform-google-modules/terraform-google-project-factory).
73+
There is an example of how to create the project [within that module](https://github.com/terraform-google-modules/terraform-google-project-factory/tree/master/examples/app_engine)
74+
75+
### Configure a Service Account
76+
In order to execute this module you must have a Service Account with the following roles.
77+
78+
- roles/storage.admin
79+
- roles/pubsub.editor
80+
- roles/cloudscheduler.admin
81+
- roles/cloudfunctions.developer
82+
- roles/iam.serviceAccountUser
83+
84+
85+
### Enable API's
86+
In order to operate with the Service Account you must activate the following API on the project where the Service Account was created:
87+
88+
- Cloud Scheduler API - cloudscheduler.googleapis.com
89+
- Cloud PubSub API - pubsub.googleapis.com
90+
- Cloud Functions API - cloudfunctions.googleapis.com
91+
92+
## Install
93+
94+
### Terraform
95+
Be sure you have the correct Terraform version (0.11.x), you can choose the binary here:
96+
- https://releases.hashicorp.com/terraform/
97+
98+
## Testing and documentation generation
99+
100+
### Requirements
101+
- [docker](https://docker.com)
102+
- [terraform-docs](https://github.com/segmentio/terraform-docs/releases) 0.6.0
103+
104+
### Integration test
105+
##### Terraform integration tests
106+
It is recommended to to run the integration tests via docker. To do so, run `make test_integration_docker`. In containers, this will
107+
- Perform `terraform init` command
108+
- Perform `terraform get` command
109+
- Perform `terraform validate` command
110+
- Perform `terraform apply -auto-approve` command and check that it has created the appropriate resources
111+
- Perform `terraform destroy -force` command and check that it has destroyed the appropriate resources
112+
113+
### Autogeneration of documentation from .tf files
114+
Run
115+
```
116+
make generate_docs
117+
```
118+
119+
### Linting
120+
The makefile in this project will lint or sometimes just format any shell,
121+
Python, golang, Terraform, or Dockerfiles. The linters will only be run if
122+
the makefile finds files with the appropriate file extension.
123+
124+
All of the linter checks are in the default make target, so you just have to
125+
run
126+
127+
```
128+
make -s
129+
```
130+
131+
The -s is for 'silent'. Successful output looks like this
132+
133+
```
134+
Running shellcheck
135+
Running flake8
136+
Running gofmt
137+
Running terraform validate
138+
Running hadolint on Dockerfiles
139+
Test passed - Verified all file Apache 2 headers
140+
```
141+
142+
The linters
143+
are as follows:
144+
* Shell - shellcheck. Can be found in homebrew
145+
* Python - flake8. Can be installed with 'pip install flake8'
146+
* Golang - gofmt. gofmt comes with the standard golang installation. golang
147+
is a compiled language so there is no standard linter.
148+
* Terraform - terraform has a built-in linter in the 'terraform validate'
149+
command.
150+
* Dockerfiles - hadolint. Can be found in homebrew

0 commit comments

Comments
 (0)